Method and system to modify geolocation activities based on logged query information

ABSTRACT

A method and a system perform geolocation activities relating to a network address. A database of network addresses, and associated geographic locations, is maintained. A query, including a network address, is received against the database for a geographic location associated with the network address. Information, concerning the query received against the database, is logged. Geolocation activities relating to at least the network address are modified based on the logged information.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.09/825,675 filed on Apr. 3, 2001, now U.S. Pat. No. 6,684,250, which inturn claims the priority benefit of U.S. Provisional Application No.60/194,761, filed Apr. 3, 2000, and U.S. Provisional Application No.60/241,776 filed Oct. 18, 2000. Each of the identified applications ishereby incorporated by reference.

FIELD OF THE INVENTION

The present invention relates generally to the field of geographiclocation determination and, more specifically, to a method and apparatusfor estimating the geographic location of a network entity, such as anode coupled to the Internet.

BACKGROUND OF THE INVENTION

Geography plays a fundamental role in everyday life and effects, forexample, of the products that consumers purchase, shows displayed on TV,and languages spoken. Information concerning the geographic location ofa networked entity, such as a network node, may be useful for any numberof reasons.

Geographic location may be utilized to infer demographic characteristicsof a network user. Accordingly, geographic information may be utilizedto direct advertisements or offer other information via a network thathas a higher likelihood of being the relevant to a network user at aspecific geographic location.

Geographic information may also be utilized by network-based contentdistribution systems as part of a Digital Rights Management (DRM)program or an authorization process to determine whether particularcontent may validly be distributed to a certain network location. Forexample, in terms of a broadcast or distribution agreement, certaincontent may be blocked from distribution to certain geographic areas orlocations.

Content delivered to a specific network entity, at a known geographiclocation, may also be customized according to the known geographiclocation. For example, localized news, weather, and events listings maybe targeted at a network entity where the geographic location of thenetworked entity is known. Furthermore content may be presented in alocal language and format.

Knowing the location of network entity can also be useful in combatingfraud. For example, where a credit card transaction is initiated at anetwork entity, the location of which is known and far removed from ageographic location associated with a owner of credit card, a creditcard fraud check may be initiated to establish the validity of thecredit card transaction.

SUMMARY OF THE INVENTION

According to one aspect of the present invention, there is provided amethod to perform geolocation activities relating to a network address.A database of network addresses and associated geographic locations ismaintained. A query, including a network address, is received againstthe database for a geographic location associated with the networkaddress. Information, concerning the query received against thedatabase, is logged. Geolocation activities relating to at least thenetwork address are modified based on the logged information.

Other aspects of the present invention will be apparent from theaccompanying drawings and from the detailed description that follows.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention is illustrated by way of example and notlimitation in the figures of the accompanying drawings, in which likereferences indicate similar elements and in which:

FIG. 1A is a diagrammatic representation of a deployment of ageolocation system, according to an exemplary embodiment of the presentinvention, within a network environment.

FIG. 1B is a block diagram providing architectural details regarding ageolocation system, according to an exemplary embodiment of the presentinvention.

FIG. 2 is a block diagram illustrating software architecture for ageolocation system, according to an exemplary embodiment of the presentinvention.

FIG. 3 is a flowchart illustrating a method, according to an exemplaryembodiment of the present invention, of collecting data utilizing anumber of data collection agents.

FIG. 4A is a state diagram illustrating general dataflow within thegeolocation system, according to an exemplary embodiment of the presentinvention.

FIG. 4B is a state diagram illustrating dataflow, according to anexemplary embodiment of the present invention, during a geolocation datacollection and analysis process.

FIG. 5 is a diagrammatic overview of dataflow pertaining to a datawarehouse, according to an exemplary embodiment of the presentinvention.

FIG. 6 is a flowchart illustrating operation of a data collection agent,according to an exemplary embodiment of the present invention, uponreceipt of a request from an associated data collection broker.

FIG. 7 is a flowchart illustrating operation of a data collectionbroker, according to an exemplary embodiment of the present invention,upon receipt of a job request from a user via an interface.

FIG. 8 is a diagrammatic representation of operation of an analysismodule, according to an exemplary embodiment of the present invention.

FIGS. 9A and 9B show a flowchart illustrating a method, according to anexemplary embodiment of the present invention, of tiered estimation of ageolocation associated with a network address.

FIGS. 10A and 10B illustrate exemplary networks, a first of which hasnot been subnetted, and a second of which has been subnetted.

FIG. 11 is a block diagram illustrating a process flow for a unifiedmapping process, according to an exemplary embodiment of the presentinvention.

FIGS. 12A and 12B illustrate respective one-dimensional andtwo-dimensional confidence maps, according to exemplary embodiments ofpresent invention.

FIG. 13 is a flowchart illustrating a method, according to an exemplaryembodiment of the present invention, performed by a RegEx LDM toidentify one or more geographic locations associated with networkaddress and associated at least one confidence factor with each of theidentified geographic locations.

FIGS. 14A-14Q illustrate an exemplary collection of confidence maps thatmay be utilized by the RegEx LDM to attach confidence factors tolocation determinants.

FIG. 15 is a flowchart illustrating a method, according to an exemplaryembodiment of the present invention, performed by the Net LDN toidentify one or more geographic locations for a network address, or ablock of network addresses, and to associated at least one confidencefactor with each of the geographic locations.

FIGS. 16A-16E illustrate an exemplary collection of confidence maps thatmay be utilized by the Net LDM to attach confidence factors to locationdeterminants.

FIG. 17 is a flowchart illustrating a method, according to an exemplaryembodiment of the present invention, performed by the DNS LDM identifyone or more geographic locations for network address, and to associatedat least one confidence factor with each of the geographic locations.

FIGS. 18A-18E illustrate an exemplary collection of confidence maps thatmay be utilized by the DNS LDM to attach confidence factors to locationdeterminants.

FIGS. 19A-19E illustrate an exemplary collection of confidence maps thatmay be utilized by the ASN LDM to attach confidence factors to locationdeterminants.

FIGS. 20A-20C illustrate an exemplary collection of confidence maps thatmay be utilized by the LKH LDM to attach confidence factors to locationdeterminants.

FIGS. 21A-21C illustrate an exemplary collection of confidence maps thatmay be utilized by the NKH LDM to attach confidence factors to locationdeterminants.

FIG. 22 is a flowchart illustrating a method, according to an exemplaryembodiment of the present invention, performed by a sandwich LDM toidentify one or more geographic locations for a network address, and toassociate at least one confidence factor with each of the geographiclocations.

FIG. 23 illustrate an exemplary confidence that may be utilized by thesandwich LDM to attach confidence factors to location determinants.

FIG. 24 is a flowchart illustrating a method, according to an exemplaryembodiment of the present invention, of filtering location determinantsreceived from a collection of LDMs utilizing a filter locationdeterminants.

FIG. 25 is a flowchart illustrating a method, according to an exemplaryembodiment of the present invention, performed by a location synthesisprocess to deliver a single location determinant that the unifiedmapping process has identified as a best estimate of a geographiclocation.

FIG. 26 is a graph illustrating correctness of location determinants, asa function of a post-location synthesis process confidence factor.

FIG. 27 is a graph illustrating correctness of location determinants asa function of post-location synthesis process confidence factor, and asmoothed probability of correctness given a confidence factor range.

FIG. 28 is a graph illustrating correctness of location determinants asa function of a post-location synthesis process confidence factor, and asmoothed probability of correctness given a confidence factor range.

FIG. 29 is a graph illustrating correctness of location determinants asa function of a post-confidence accuracy translation confidence factor,and a smoothed probability of correctness.

FIG. 30 shows a diagrammatic representation of a machine in exemplaryform of a computer system within which a set of instructions, forcausing the machine to perform any of the methodologies discussed above,may be executed.

The file of this patent contains at least one drawing executed in color.Copies of this patent with color drawing(s) will be provided by thePatent and Trademark Office upon request and payment of the necessaryfee.

DETAILED DESCRIPTION

A method and system to modify geolocation activities based on loggedquery information are described. In the following description, forpurposes of explanation, numerous specific details are set forth inorder to provide a thorough understanding of the present invention. Itwill be evident, however, to one skilled in the art that the presentinvention may be practiced without these specific details.

For the purposes of the present specification, the term “geographiclocation” shall be taken to refer to any geographic location or areathat is identifiable utilizing any descriptor, metric or characteristic.The term “geographic location” shall accordingly be taken to include acontinent, a country, a state, a province, a county, a city, a town,village, an address, a Designated Marketing Area (DMA), a MetropolitanStatistical Area (MSA), a Primary Metropolitan Statistical Area (PMSA),location (latitude and longitude), zip or postal code areas, andcongressional districts. Furthermore, the term “location determinant”shall be taken to include any indication or identification of ageographic location.

The term “network address”, for purposes of the present specification,shall be taken to include any address that identifies a networkedentity, and shall include Internet Protocol (IP) addresses.

Typically, most network addresses (e.g., IP addresses) are associatedwith a particular geographic location. This is because routers thatreceive packets for a particular set of machines are fixed in locationand have a fixed set of network addresses for which they receivepackets. The machines that routers receive packets for tend to begeographically proximal to the routers. Roaming Internet-Ready devicesare rare exceptions. For certain contexts, it is important to know thelocation of a particular network address. Mapping a particular networkaddress to a geographic location may be termed “geolocation”. Anexemplary system and methodology by which geographic locations can bederived for a specific network addresses, and for address blocks, aredescribed below. Various methods of obtaining geographic information,combining such geographic information, and inferring a “block” to whicha network address corresponds and which shares the same geographicinformation are described.

The exemplary system and method described below include (1) a datacollection stage, (2) a data analyses stage, and (3) a delivery stage.

System Architecture

FIG. 1A is a diagrammatic representation of a deployment of ageolocation system 10, according to an exemplary embodiment of thepresent invention, within a networked environment 8. Various componentsof the system 10 are shown in the attached Figures to be coupled bynetworks 4. The geolocation system 10 is shown to include: (1) a datacollection and analysis system 12 that is responsible for the collectionand analysis of information useful in geolocating a network address; (2)a delivery engine system 16, including a number of delivery engineservers 64, which operate to provide geolocation information to acustomer; and (3) a data warehouse 30 that stores collected informationuseful for geolocation purposes and determining geolocations forspecific network addresses (or blocks of network addresses).

Geolocation data is distributed from the data warehouse 30 to thedelivery engine system 16 for delivery to a customer in response to aquery.

More specifically, in one exemplary embodiment, the data collection andanalysis system 12 operates continuously to identify blocks of networkaddresses (e.g., Class B or Class C subnets) as will be described infurther detail below, and to associate a geographic location(geolocation) with the identified blocks of network addresses. A recordis then written to the data warehouse 30 for each identified block ofnetwork addresses, and associated geolocation. In one exemplaryembodiment, a record within the data warehouse 30 identifies a block ofnetwork addresses utilizing a subnet identifier. In a further exemplaryembodiment, a record within the data warehouse identifies a start andend network address for a relevant block of network addresses. In aneven further exemplary embodiment, a record identifies only a singlenetwork address and associated geolocation. The data collection andanalysis system 12 operates to continually updated and expand thecollection of records contained within the data warehouse 30. Anadministrator of the data collection and analysis system 12 mayfurthermore optionally directed the system 12 to focus geolocationactivities on a specific range of network addresses, or to prioritizegeolocation activities with respect to specific range of networkaddresses. The data collection and analysis system 12 furthermoremaintains a log of network addresses received that did not map to ablock of network addresses for which a record exists within the datawarehouse 30. The data collection and analysis system may operate toprioritize geolocation activities to determine geolocation informationfor network addresses in the log.

In an exemplary use scenario, an Internet user may, utilizing a usermachine that hosts a browser 1, access a web site operated by thecustomer. The custom website is supported by the application server 6,which upon receiving an IP address associated with the user machine 2,communicates this IP address to the geolocation Application ProgramInterface (API) 7 hosted that the customer site. Responsive to receivingthe IP address, the API 7 communicates the IP address to a deliveryengine server 64 of the delivery engine system 16.

In the manner described in further detail below, the data collection andanalysis system 12 generates a location determinant, indicating at leastone geographic location, and an associated location probability table,that is communicated back to the customer. More specifically, thedelivery engine server 64 attempts to identify a record for a block ofnetwork addresses to which the received IP address belongs. If thedelivery engine server 64 is successful in locating such a record,geolocation information (e.g., a location determinant) stored withinthat record is retrieved and communicated back to the customer. On theother hand, if the delivery engine server 64 is unsuccessful in locatinga record within the data warehouse 30, the relevant IP address islogged, and a “not found” message is communicated to the customerindicating the absence of any geolocation information for the relevantIP address.

The customer is then able to utilize the location determinant for anyone of multiple purposes (e.g., targeted advertising, contentcustomization, digital rights management, fraud detection etc.)

FIG. 1B is a block diagram providing further details regarding aphysical architecture for the geolocation system 10, according to anexemplary embodiment of the present invention. At a high level, thegeolocation system 10 comprises the data collection and analysis system12, a data warehouse system 14, and the delivery engine system 16. FIG.2 is a block diagram illustrating software architecture for thegeolocation system 10, according to an exemplary embodiment of thepresent invention.

The data collection and analysis system 12 is shown to collect data fromgeographically dispersed, strategically placed remote data collectionagents 18, hosted on data collection machines 20. A group of datacollection agents 18 is controlled by a data collection broker 22, whichmay be hosted on a data analysis server 24. The data collected by a datacollection broker 22, as shown in FIG. 2, is delivered to a datacollection database 26, and is analyzed utilizing an analysis module 28.The analysis module 28 implements a number of analysis techniques toattach a known or estimated geographic location to certain networkinformation (e.g., the source or destination address of a networkrequest). A resulting location record, along with all supportinginformation, is then written into a data warehouse 30 of the datawarehouse system 14. The geolocation system 10, in one embodiment,supports the following features:

Implementation of a data collection agent 18 capable of individuallyperforming a number of data collection operations in accordance with anumber of analysis techniques utilized by the analysis module 28; and

Implementation of a data collection broker 22 capable of determiningwhich of a number of analysis techniques, utilized by the analysismodule 28, to utilize for a given network information (e.g., an IPaddress).

FIG. 2 illustrates a number of a data collection agents 18 hosted atgeographically disperse locations. For example, these disperse locationsmay be with separate service providers. The location of the datacollection agents 18 at disperse locations assists the geolocationsystem 10 by providing different “points of view” on the network target.

Each data collection agent 18 is responsible for actual execution of adata collection process, or search, to locate and extract data that isthe useful for the determination of a geolocation. Further detailsregarding exemplary searches are provided below. For example, atraceroute search is conducted by a data collection agent 18 responsiveto a search request received at a data collection agent 18 from a datacollection broker 22. Each data collection agent 18, responsive to arequest, will perform a search (e.g., a traceroute) to collect specifieddata, and determine the validity of the raw data utilizing built-inmetrics. If successful, this data is provided to the data collectiondatabase 26, via a data collection broker 22, for analysis by theanalysis module 28. Each data collection agent 18 further advises acontrolling data collection broker 22 of the success or failure of aparticular search.

Each data collection broker 22 controls a group of data collectionagents 18. For example, given a network address, or a range of networkaddresses, a data collection broker 22 determines which data collectionagents 18 are most appropriate for the specific search. Once the requesthas been sent to a group of data collection agents 18 from a datacollection broker 22, a response is expected containing a summary of thesearch. If the search was successful, this information will be placeddirectly into the data collection database 26, at which time theanalysis module 28 will determine an estimated geolocation of thesearched addresses.

On the other hand, if a search is not successful, the data collectionbroker 22 takes the appropriate action, and the data is not entered intothe data collection database 26. At this time, the data collectionbroker 22 hands the search request to another data collection broker 22,which performs the same process.

The data collection database 26 contains current state information, aswell as historical state information. The state information includesstatistics generated during the data acquisition by the data collectionagents 18, as well as failure statistics. This allows an operator of thegeolocation system 10 to visualize the actual activity of a datacollection process.

Data Collection

FIG. 3 is a flowchart illustrating a method 38, according to anexemplary embodiment of the present invention, of collecting datautilizing a number of data collection agents 18.

At block 40, a user (or process) enters a job request to the datacollection broker 22 via, for example, a web interface. Job schedulingis also an option for the user. At block 42, the relevant datacollection broker 22 accepts a request, and determines what datacollection agents 18 will service the request. The data collectionbroker 22 also sets a unique session identifier (USID).

At block 44, one or more data collection agents 18 accept a job, andreport to the data collection broker 22 that submission was successful.

At block 46, the data collection broker 22 writes (1) a start mark,indicating that the job is underway, and (2) the unique sessionidentifier to the data collection database 26.

At block 48, the data collection agents 18 perform various searches(e.g., traceroutes) to collect raw data, and stores results locally forlater batch update.

At block 50, each of the data collection agents 18 informs the datacollection broker 22 that the search has finished, with or withoutsuccess. After the last data collection agent 18 reports its status, thedata collection broker 22 instructs the data collection agents 18 toupload their information to the data collection database 26.

At block 52, after the last data collection agent 18 reports a finisheddatabase write, the data collection broker 22 instructs the datacollection agents 18 to flush their local storage, and remain idle untilthe next search job.

At block 54, the analysis module 28 processes the newly entered datawithin the data collection database 26, and writes this data to the datawarehouse 30.

The delivery engine system 16 is responsible for delivering geolocationinformation generated by the geolocation system 10. With reference toFIG. 1, the delivery engine system 16 may be viewed as comprising adelivery staging server 60, a statistics processing engine 62, one ormore delivery engine servers 64 and a delivery engine plant daemon (notshown).

The delivery staging server 60 provides a reliable and scaleablelocation distribution mechanism for geolocation data and does not modifyany data. The delivery staging server 60 provides a read-only copy ofthe geolocation information to the delivery engine servers 64, and isresponsible for preparing geolocation information that should bedistributed to the delivery engine servers 64. Each delivery stagingserver 60 prepares dedicated information for one product offering. Thedelivery staging server 60 will retrieve the geolocation informationfrom the data warehouse 30 based on the product offering. The deliverystaging server 60 configuration includes a customer list and a deliveryengine servers list for deployment. At fixed intervals, geolocationinformation is refreshed from the data warehouse 30 and distributed tothe delivery engine service 64. The refresh from the data warehouse 30may be based on a number of factors such as a new product offering orrefining the existing location data. Before each new load of thedelivery engine servers 64, the delivery staging server 60 retrieves acurrent copy of customers and the delivery engine servers 64 associatedwith the relevant delivery staging server 60.

The administration of the delivery staging servers 60 is performed by aseparate server that is also responsible for load balancing and backupconfiguration for the delivery staging servers 60.

The statistics processing engine 62 is responsible for retrievingcustomer access logs (hits and misses) and usage data from the deliveryengine services 64 on a regular basis. This information is used, forexample, as input for the load balancing criteria, and getting updateinformation for the location misses. The usage statistics may alsoprovide the required information to the billing subsystem.

All information sent to delivery engine service 64 is encrypted toprevent unauthorized use.

The delivery engine servers 64 are responsible for serving the clientsof the geolocation system 10. The delivery engine servers 64 may behosted at a client site or at a central data center. The delivery engineservers 64 are able to accept update information from the deliverystaging server 60 and to serve current requests. Each delivery engineservers 64 saves all customer access information and provide thisinformation to the statistics processing engine 62. In embodiment, eachdelivery engine server 64 provides an eXtensible Markup Language(XML)-based Application Program Interface (API) interface to thecustomers of the geolocation system 10.

A geolocation API 7, as described above with reference FIG. 1A,interfaces with a delivery engine server 64 from a customer applicationserver. The geolocation API 7 may support a local cache to speed up theaccess, this cache being flushed whenever the delivery engine server 64is reloaded. The geolocation API 7 may be configured to access analternate server in case of a failure or high load on a single deliveryengine server 64. Each delivery engine server 64 and delivery stagingserver 60 includes a Simple Network Management Protocol (SNMP) agent fornetwork management.

Data Flow (Collection, Analysis and Delivery)

FIG. 4A is a state diagram illustrating general data flow, as describedabove and according to an exemplary embodiment of the present invention,within the geolocation system 10. FIG. 4B is a state diagramillustrating data flow, according to an exemplary embodiment of thepresent invention, during the geolocation data collection and analysisprocesses described above.

The analysis module 28 retrieves geolocation information from the datacollection database 26 to which all data collection agents 18 write suchinformation, in the manner described above. Specifically, the analysismodule 28 operates a daemon, polling in a timed interval for new datawithin the data collection database 26. When new data is found, theanalysis techniques embodied within sub-modules (Location DeterminationModules LDMs) of the analysis module 28 are initiated, with the resultsof these analysis techniques being written to the primary data warehouse30.

FIG. 5 provides an overview of data flow pertaining to the datawarehouse 30, according to an exemplary embodiment of the presentinvention. As described above, data collection is performed by the datacollection and analysis system 12. The results of the collection processare aggregated in the data collection database 26, which is anintermediary datastore for collection data. At some later point, data istaken from the database 26 by the analysis module 28, and the finalanalysis, along with all the supporting data, is placed into the datawarehouse 30. The delivery staging servers 16 then pull a subset of datafrom the data warehouse 30 (this defines a product offering), and placethis information into a staging database (not shown) associated with thedelivery staging server 60. A staging database then pushes a copy of thegeolocation information out to all delivery engine servers 64, which runa particular product offering.

The delivery staging servers 60 may provide the following customerinformation to the data warehouse 30:

-   -   Customer Registration    -   Customer Product License—level of support.        The following data is outputted from the data warehouse 30:    -   Product Description (US, whole Europe, UK etc)    -   Get customer list for the given product type    -   Get location information for the product.    -   Get list of delivery engine servers 64 that map to the product        offering.    -   Store location data on the disk with version number    -   Build an in-memory database    -   Create customer specific information from the memory database.    -   Transfer data to Delivery Engine Production Systems.        The delivery staging servers 60 process requests from a client        application by:    -   Parsing XML requests received from a client application.    -   Logging requests.    -   Looking up location information based on level of service.    -   Constructing a response and communicating the response back to        the client application.

The delivery staging servers 60 process database updates by storing anew database with a version number on disk and building a new in-memorydatabase for updates. Each update is a complete replacement of theexisting in-memory database

The statistics processing engine 62 activates after a given period oftime, checks the data warehouse 30 for a list of active client machines,and retrieves the statistics files from all of the deployed deliveryengine servers 64. Once such files have been retrieved, the statisticsprocessing engine 62 pushes the statistics into the data warehouse 30.

The geolocation system 10, according to the one embodiment, utilizeseXtensible Markup Language (XML) as a data transfer format, both withinthe above-mentioned subsystems, and as the delivery agent to customersystems. XML offers flexibility of format when delivering geolocationinformation, and extensibility when the geolocation system 10 offersextended data in relation to geographic location, without having toreprogram any part of the client interfaces.

A standard XML parser technology may be deployed throughout thegeolocation system 10, the parser technology comprising either theXerces product, a validating parser offered by the Apache group, or XMLfor C++, written by the team at IBM's AlphaWorks research facility,which is based on the Xerces parser from Apache, and includes Unicodesupport and other extensions.

The geolocation system 10 utilizes numerous Document Type Definitions(DTDs) to support the XML messaging. DTDs serve as templates for validXML messages.

The standard response to a customer system that queries the geolocationsystem 10, in one exemplary embodiment of the present invention, is inthe form of a location probability table (LPT), an example of which isprovided below. A location probability table may be an XML formattedmessage, containing a table of information representing locationgranularity (or resolution), location description, and a confidencepercentage.

<?xml version=“1.0”?> <Service provider name>   <geolocationtype=“response”>     <ipaddress>128.52.46.11</ipaddress>     <lpt>        <continent>           <value type=“string”>North America</value>          <confidence>100%</confidence>       </continent>        <country>           <value type=“string”>United States</value>          <confidence>99%</confidence>       </country>         <region>          <value type=“string”>New England</value>          <confidence>97%</confidence>       </region>         <state>          <value type=“string”>Massachusetts</value>          <confidence>96%</confidence>       </state>         <areacode>          <value type=“integer”>617</value>          <confidence>94%</confidence>       </areacode>         <msa>          <value type=“string”>Boston MSA</value>          <confidence>94%</confidence>       </msa>         <city>          <value type=“string”>Cambridge</value>          <confidence>93%</confidence>       </city>         <zipcode>          <value type=“integer”>02142</value>          <confidence>91%</confidence>       </zipcode>     </lpt>  </geolocation> </quova>

As will be noted from the above example, the location probability tableindicates multiple levels of geographic location granularity orresolution, and provides a location probability (or confidence factor)for each of these levels of geographic resolution. For example, at a“country” level of geographic resolution, a relatively high probabilitylevel may be indicated. However, at a “city” level of geographicresolution, a relatively low probability level may be indicated in viewof a lower confidence in the geolocation of the network entity at anindicated city.

The above location probability table constitutes a XML response to ageolocation request for the IP address 128.52.46.11. The city where theaddress is located is Cambridge, Mass., USA, identified with granularity(or geographic resolution) down the zip code level, at a 91% confidence.

In an alternative embodiment, the location probability table may beformatted according to a proprietary bar delimited format specification.

A more detailed description of the various systems that constitute thegeolocation system 10, and operation of the geolocation system 10, willnow be provided.

A data collection agent 18 operates to receive commands from anassociated data collection broker 22, and includes logic to execute anumber of data collection operations specific to a number of analysisprocesses implemented by the analysis module 28. Each data collectionagent 18 reports results back to an associated data collection broker 22that performs various administrative functions (e.g., start, stop,restart, load, process status). FIG. 6 is a flowchart illustratingfunctioning of a data collection agent 18, according to an exemplaryembodiment of the present invention, upon receipt of a request from anassociated data collection broker 22.

A data collection broker 22 determines what actions are requiredresponsive to a request from a customer (e.g., check new addresses,recheck older addresses, etc.), and provides instructions to one or moredata collection agents 18 regarding what function(s) to perform withrespect to certain network information (e.g., a network address).

Each data collection broker 22 further stores raw data (geolocationinformation) into the data collection database 26, performs loadbalancing of requests across multiple data collection agents 18,performs administrative functions with respect to data collection agents18 (e.g., requests stops, starts, status etc.) and performs variousinternal administrative functions (e.g. start, stop, restart, load).FIG. 7 is a flowchart illustrating functioning of a data collectionbroker 22, according to an exemplary embodiment of the presentinvention, upon receipt of a job request from a user via a Web interfaceor any other interface.

The analysis module 28, according to one exemplary embodiment, operatesto extract raw data from the data collection database 26, process thedata according to one or more analysis algorithms (or modules) togenerate a location probability table, and to store results and the rawdata into the data warehouse 30. FIG. 8 is a diagrammatic representationof operation of the analysis module 28, according to an exemplaryembodiment of the present invention.

The delivery engine servers 64 except queries (e.g., in XML format),return responses, lookup query information in a main memory database,report statistics to flat files for the data processing, respond toadministrative functions, and except push updates to create secondrun-time databases and perform switchover.

The delivery servers 64 operate to scan content within the datawarehouse 30, creating specific service offerings (e.g., North America,by continent, by country), and push content out to the delivery engineservers 64.

Data Collection

As described above, each of the data collection agents 18 may implementone of multiple data collection processes to obtain raw geolocationinformation. These data collection processes may, in one exemplaryembodiment of the present invention, access any one or more of thefollowing data sources:

Net Whois Record: The Net Whois record is an entry in a registry thattracks ownership of blocks of Internet Protocol (IP) addresses andaddress space. Such records are maintained by RIPE (Reseaux IPEuropeens), APNIC (Asia Pacific Network Information Centre), ARIN(American Registry of Internet Numbers), and some smaller regionalInternet registries. For instance, the IP network address 192.101.138.0is registered to Western State College in Gunnison, Colo.

DNS Whois Record: The DNS Whois record is an entry in a registry thattracks ownership of domain names. This is maintained by NetworkSolutions, Inc. For instance, quova.com is registered to Quova, Inc. inMountain View, Calif.

ASN Whois Record: An ASN Whois record is an entry in a registry thattracks autonomous systems. An autonomous system (AS) is a collection ofrouters under a single administrative authority using a common BorderGateway Protocol for routing packets. ASN databases are maintained by anumber of organizations.

DNS Loc Record: Occasionally, a DNS Location (Loc for short) record isstored, which indicates the precise latitude, longitude, and elevationof a host.

Traceroute: A traceroute shows the route of a data packet from a datacollection machine to a target host. Much information can be derivedfrom the analysis of a traceroute. For instance, if hop #10 is inCalifornia, and hop #13 is in California, then with increased certainty,it can be inferred that hops #11 and #12 are also in California.

In addition to the above data that may be collected by the datacollection agents 18, the analysis module 28 may also utilize thefollowing information sources in performing an analysis to estimate ageographic location for network address:

Hostname: An IP network address is often tied to a hostname. Thehostname may have information indicative of location. Carriers typicallyimplement this to more easily locate their own hardware. For instance,bbr-g2-0.sntc04.exodus.net is in Santa Clara, Calif.; ‘sntc’ is Exodus'abbreviation for Santa Clara.

Demographic/Geographic Data: Implicit in much of the decision makingprocesses is information about the different locations of the world. Theanalysis module 28, in one embodiment, utilizes a demographic/geographicdatabase 31, shown in FIGS. 1B and 2 to be part of the data warehouse30, storing a city record for every city in the U.S.A. and all foreigncities with populations of greater than 100,000 people. Tied to eachcity are its state, country, continent, DMA (Designated Marketing Area),MSA (Metropolitan Statistical Area), PMSA (Primary MetropolitanStatistical Area), location (latitude & longitude), sets of zip/postalcodes, congressional districts, and area codes. Each city record alsohas population and a connectivity index, which is based on the number ofmajor carriers that have presence in that city.

Analysis Module

As illustrated in FIG. 2, the analysis module 28 includes a collectionof blocking algorithms 63, a unified mapping process 61, and aconsolidated domains algorithm 65. FIGS. 9A and 9B show a flowchartillustrating a method 70, according to an exemplary embodiment of thepresent invention, of tiered estimation of the geolocation associatedwith a network address. Specifically the tiered estimation of ageolocation employs a number of exact processes and, if the exactprocesses fail, a number of inexact processes. In an alternativeembodiment of the present invention, no distinction is made betweenexact and inexact processes (as shown in FIG. 11), and all processes areregarded as being located on a common tier. The method 70 is performedby the analysis module 28, and employs each of the algorithms 61, 63 and65.

The method 70 commences at block 72 with the obtaining of a networkaddress (e.g., an IP address) to be mapped. This network address may bereceived from an internal process performing an automated mappingoperation (e.g., updating the geolocation information associated with aspecific IP address), or from an external source (e.g., a customer thatrequires geolocation information concerning an IP address). The obtainednetwork address is then queued within a main queue.

At block 74, the consolidated domain algorithm 65 is run. Specifically,a network address is removed from the main queue, and tested todetermine whether it is likely to fall within a consolidated domain. Ifthe tests of satisfied, as determined at decision block 76, the relevantnetwork address and the geolocation information determined by theconsolidated domains algorithm 65 are written to a record within thedata warehouse 30 at block 78.

The consolidated domain algorithm 65 utilizes the fact that some domainshave all of their IP network addresses concentrated in a singlegeographic location. The domain suitability is judged by the algorithm65 on the basis of other domain properties other than size. Such domainstypically include colleges and universities (except those that havemultiple campuses), small businesses that are known to be located in asingle location, government labs, etc.

Examples of domains that may be utilized by the algorithm 65 include:

The “.edu” domain: Because of the nature of educational institutions,“.edu” domains are typically consolidated domains. An extensive list of“.edu” domains can be obtained from web resources (by looking up theappropriate categories under the main search engines). IP lists (fromweb-server access logs, etc.) can also be translated to names andchecked for an ending “.edu”. Then they can be sorted into unique names.

Local businesses: The major web search engines also list localbusinesses for each area.

Local Internet Service Providers (ISPs): Some Internet Service Providersare local to only one region.

Government laboratories: A number of government laboratories satisfy theconsolidated domain criterion.

The above described method may encounter domain names that containextraneous information (e.g., “glen.lcs.mit.edu”), when in fact thedomain name required is “mit.edu”. In general, the name behind the“.edu.” entry is part of the domain but everything before it isextraneous (note that this will include .edu domains in othercountries). This also holds for government labs (“x.gov”), andcommercial (“x.com”). Names derived from the above methods arepre-processed to truncate them to the appropriate domain name accordingto the above rules.

Returning to FIG. 9A, if the conditions of the consolidated domainalgorithm remain unsatisfied, at block 80, the relevant network addressis reinserted into the main queue, and flagged as having failed tosatisfy the conditions imposed by the consolidated domain algorithm 65.

At block 82, one or more of blocking algorithms 63 are executed todetermine a network address block size around the relevant networkaddress. Further details regarding exemplary blocking algorithms 63 areprovided below. A blocking algorithm 63 performs a check of neighboringnetwork addresses to find the expanse of a “block” of network addressesthat share common information (e.g., a common subnet segment). Theidentification of a block of network addresses is useful in thatinformation regarding a particular network address may often be inferredfrom known information regarding neighboring network addresses within acommon block.

At block 84, if a block of network addresses associated with a subjectnetwork address is identified, this block of network addresses is theninserted into the main queue for further processing in association withthe subject network address.

Moving on to FIG. 9B, at block 86, one or more “exact” geographiclocation processes (e.g., traceroutes, latency calculations, hostnamematching and the DNS Loc LDM) are run to determine whether geolocationinformation can be determined for the subject network address, andoptionally for other network addresses of the block of networkaddresses. The “exact” processes are labeled as such as they rendergeolocation information with a relatively high confidence factor.Further, the exact processes may render geolocation information forneighboring network addresses within a block to increase the confidencefactor of geolocation information rendered for a subject networkaddresses.

At decision block 88, a determination is made as to whether the exactprocesses with successful in generating geolocation information with apredetermined confidence factor, and whether a blocking was verified. Ifso, at block 90, the network address and the determined geolocationinformation are written into a record within the data warehouse 30.

On the other hand, following a negative determination at decision block88, the method 70 progresses to block 92, where a series of “inexact”geographic location operations (or algorithms) are executed on thesubject network address, and optionally on one or more network addresseswithin an associated block. The “inexact” processes are labeled as suchin view of the relatively lower confidence factor with which theseinexact processes render geolocation information associated with anetwork address. In one exemplary embodiment, a number of inexactprocesses are executed on a number of network addresses surrounding asubject network address, and the outputs of these inexact processes areconsolidated by the unified mapping process 61, which considers theoutput from each of the number of inexact processes (e.g., the belowdiscussed Location Determination Modules (LDMs)). Further details areprovided below.

At decision block 94, a determination is made as to whether the inexactprocesses generated a predetermined confidence factor for geolocationinformation for the subject network address. If so, the network addressand associated geolocation information are again written into a recordwithin the data warehouse 30 at block 96. On the other hand, following anegative determination at decision block 94, the network address may beforwarded for a manual resolution at block 98. The method 70 then exitsat block 100.

Queuing

Queuing interfaces exist for both processes (e.g., scripts oralgorithms) scripts that enter items into the main queue discussedabove, as well as for processes that remove items form the main queue.

When a “block” of network addresses is successfully entered into thedata warehouse 30 by an exact algorithm at block 90, the entire mainqueue is searched for entries that fall within that block of networkaddresses. These entries are then be removed because they are part of ablock that is known to be accurate. If a block of network addresses isentered the data warehouse 30 with a high confidence factor, the mainqueues are searched for entries within that block. These entries canthen be forwarded to a quality assurance queue (not shown).

Blocking

As stated above, one or more blocking algorithms 63 are executed atblock 82 shown in FIG. 9A to identify a “block” of network addressessurrounding a subject network address that may share common informationor characteristics with the subject network address. Three exemplaryblocking algorithms 63 to perform a blocking operation around a subjectnetwork address are discussed below, namely: (1) a divide-and-conquerblocking algorithm; (2) a netmask blocking algorithm; and (3) a blockingalgorithm that utilizes RTP tables, BGP tables, and ISP topology maps.As is described with reference to blocks 86 and 92, once an entirenetwork segment has been blocked, the entire network segment can beprocessed by the exact and inexact processes, and return one completerecord for each network that he stored within the data warehouse 30.This is advantageous in that the number of hosts that are required to beprocessed is reduced, and the amount of data that is required to becollected is also reduced.

The divide-and-conquer blocking algorithm receives a subject networkaddress, and possibly the associated information (e.g. location), andchecks neighboring network addresses to find the extent of the block ofnetwork addresses that share the common information. The algorithmstarts with a first test network address halfway to the end of a blockand test with a predicate to determine whether the first test networkaddress has same information as the subject network address. The“distance” between the subject network address and the first networkaddress is then halved and the result added to the current distance ifthe answer was positive, or subtracted from the current distance if theanswer was negative. This process is repeated until the distance offsetis one. The divide-and-conquer blocking algorithm then returns to thetop end of the block and, after competing an iteration, returns to thebottom end of the block.

The following exemplary Perl code implements a divide-and-conqueralgorithm on the IP network address space:

#!/usr/local/bin/perl # # Script to figure out the blocks for us givenan IP # # # Target IP is expected as the first parameter # # define acouple of simple helper procedures # sub int2ip {   local ($i, $a, $b,$c, $d);   $i = @_[0];   $a = int($i / (256*256*256));   $i = $i %(256*256*256);   $b = int($i / (256*256));   $i = $i % (256*256);   $c =int($i / 256);   $i = $i % 256;   $d = $i;   return (“$a.$b.$c.$d”); }sub ip2int {   split /\./, @_[0];   return (@_[0]*256*256*256 +      @_[1]*256*256 +       @_[2]*256 +       @_[3]); } # # Let's start!# $ip = $ARGV[0]; $ipn = ip2int($ip); # set the distance to the initialvalue and let's go $offset = 256*256*256*256 − $ipn; $offset = int($offset / 2); $dist = $offset; # do successive approximation for thetop end of the block while ($offset > 0) {   $test_ip = int2ip($ipn +$dist);   $offset = int($offset / 2);   if (test_pred($test_ip, $ipn)) {    $dist = $dist + $offset;   } else {     $dist = $dist − $offset;   }} $top = int2ip($ipn + $dist); # set the distance to the initial valueand let's go $offset = int ($ipn / 2); $dist = $offset; # do successiveapproximation for the bottom end of the block while ($offset > 0) {  $test_ip = int2ip($ipn − $dist);   $offset = int($offset / 2);   if(test_pred($test_ip, $ipn)) {     $dist = $dist + $offset;   } else {    $dist = $dist − $offset;   } } $bottom = int2ip($ipn − $dist); #$bottom and $top now contain the lower and upper bounds # of the blockrespectively

Note should be taken of the call to test_pred( ). This takes an IPnetwork address and returns true if this IP network address shares thesame information (i.e., is part of the same block) as the subject IPnetwork address.

The function of the test predicate is to discover if the new networkaddresses explored by the divide-and-conquer algorithm belong to thesame block as the subject network address. There are a number ofexemplary ways in which this test predicate can be implemented. Forexample:

Obtaining a location: The unified mapping process 61 can be run on thetest network address to derive a location and this location can bematched against the location of the subject network address. Thisimposes a relatively large-overhead per iteration of thedivide-and-conquer algorithm.

Traceroute information: If the subject network address and the testnetwork address follow the same route (modulo the last hop) then thenetwork addresses are part of the same block.

DNS service: This test renders a positive result if the test networkaddress and the subject network address use the same DNS server.

It will be appreciated that a number of other test predicates may bedevised to implement blocking.

The netmask blocking algorithm, according to an exemplary embodiment ofthe present invention, relies on the assumption that a subnet willgenerally not be spread over multiple locations. If parts of a block ofnetwork addresses are in differing locations, such network addressestypically require a long-distance line and a switch or router to handlethe traffic between locations. In such situations, it is generally moreconvenient to divide the network into a number of subnets, one for eachlocation. Subnets in effect form a lower bound on the block-size.Therefore, blocking can be performed by obtaining the netmask (andtherefore the subnet bounds) for a given network address (e.g., an IPaddress). Netmask's may be obtained from a number of sources, forexample:

Obtaining netmasks by Internal Control Message Protocol (ICMP): One ofthe ICMP control packets is a request for the netmask of a particularinterface. Normally, the ICMP specification states that an interfaceshould respond to such a packet only if the appropriate flag has beenset. However, there are a number of implementations of ICMP that arebroken so that the interface will respond promiscuously.

Obtaining netmasks by Dynamic Host Configuration Protocol (DHCP): Ondialing up to an ISP, a machine usually sends a DHCP request to obtainits network configuration information. Included in this information is anetmask. Monitoring the DHCP response (or in the case of Linux, an“ifconfig” call) will reveal this netmask. An automated script that doeseither is included in the dialup scripts to derive blocking informationas mapping by the ISP dialup method occurs. Because the subnets may besubsets of the actual block, multiple dialup sessions may have to occurbefore the complete block is revealed.

Turning now specifically to the Internet, the smallest subnet that isusable on the Internet has a 30-bit subnet mask. This allows two hosts(e.g., routers) to communicate between themselves. Below is an exampleof a Class C Network that has been subnetted with a 30-bit subnet mask:

-   -   (1) First network with a 30-bit subnet mask:    -   x.x.x.0 Network Address    -   x.x.x.1 Lowest Usable Host    -   x.x.x.2 Highest Usable Host    -   x.x.x.3 Broadcast Address    -   (2) Second network with a 30-bit subnet mask:    -   x.x.x.4 Network Address    -   x.x.x.5 Lowest Usable Host    -   x.x.x.6 Highest Usable Host    -   x.x.x.7 Broadcast Address    -   (3) Third network with a 30-bit subnet mask:    -   x.x.x.252 Network Address    -   x.x.x.253 Lowest Usable Host    -   x.x.x.254 Highest Usable Host    -   x.x.x.255 Broadcast Address

Knowing that the smallest subnet mask is a 30-bit subnet mask, thenetmask blocking algorithm can avoid “hitting” the lowest address (i.e.,the Network Address) and the highest address (i.e., the BroadcastAddress) of a subnet by stepping through the address space. Thistechnique allows the netmask blocking algorithm to avoid automaticsecurity auditing software that may incorrectly assumed a SMURF attackis being launched.

Only two hosts per subnet/network are required by the netmask blockingalgorithm to determine if it has been “subnetted” or not, provided thatthe IP network addresses are sufficiently far apart.

The below described algorithm provides at least two benefits, namely (1)that the data collection process becomes less intrusive and (2) aperformance benefit is achieved, in that by limiting the number of hoststhat are processed on each network, it is possible to “process” a largenetwork (e.g., the Internet) utilizing a relatively small data set.

Consider the example of a Class C network that is not subnetted, such asthat illustrated at 102, in FIG. 10A. This can be determined bycollecting traceroute data from a low host (e.g., less than 128) and ahigh host (e.g., greater than 128) by examining the next-to-last hop inboth traceroute's, it is observed that both trace hops go through thesame next-to-last-hop router, and therefore utilizing the same subnet.

FIG. 10B is a diagrammatic representation of a Class C network 104 thathas been subnetted. In this example, it is assumed that traceroute datafor the network addresses 2.2.2.1 and 2.2.2.254 has been collected andis known. By looking one hop back, it can be determined that the networkhas been subnetted. Since the network is identified as being subnetted,additional host will be required. For example in a Class C network, 256hosts may be divided over multiple locations. For example, IP addresses1-64 may be in Mountain View, 65-128 may be in New York, 129-and 92 maybe in Boston, and 123-256 are in Chicago. This example, even though anetwork block is registered as a Class C network to an entity, multiplerecords are required to accurately represent the data since there aremultiple locations for the entity. In this case, 4 records are required.The netmask blocking algorithm accordingly starts looking for hosts atthe high end of the lower network, and inversely for the low end of thehigh network. Assuming that responses are obtainable from the hosts inthe network illustrated in FIG. 10A, it can be determined by the subnetblocking algorithm that the Class C network has been subnetted once.Another way of determining this outcome would be to view the relevantnetwork as two 25-bit networks, rather than a single 24-bit network.

This technique of “divide and conquer”, combined with more selectivepinging/tracerouting allows the subnet blocking algorithm to create areduced impression in security logs of networks.

A further consideration is a situation in which a traceroute is obtainedto a router that has an interface on an internal network. In this case,the traceroute will stop at the routers external interface. This mayresult in the blocking of a network multiple times. In order to addressthis problem, a determination is made by the subnet blocking algorithmas to whether the end node of a traceroute is the same as thenext-to-last hop of other traceroutes on the network. If so, the abovedescribed situation is detected.

Digital Subscriber Line (DSL) and cable modems do not appear to routerswhen they have multiple interfaces. This can result in the creation offalse results. To address the situation, the subnet blocking algorithmlooks for patterns in the last three hops. By looking at thisinformation, the algorithm is able to determine appropriate blocking forthe high-speed modem network.

Some routers also allow for networks to be subnetted to different sizeswithin a predefined network block. In this situation, a Class C networkmay be subnetted into two networks, one of which is then further dividedinto number of smaller networks. To account for the situation, thesubnet blocking algorithm verifies every block within two traceroutes.This enables the location of at least one node per network.

A further exemplary algorithm may also perform blocking utilizing RIPtables, BGP tables and ISP topology maps. The division into blocks thatare routed to a common location stems from the way routing is performed.Availability of the internal routing tables for an Autonomous System, ora topology map for an ISP, may be utilized to obtain the blockinformation as such tables and maps explicitly named the blocks that arerouted through particular routes.

Using RIP and other internal routing tables: Routing tables have astandard format. Each route consists of a network prefix and possibly anetblock size, along with the route that IP addresses belonging to thatnetblock should follow and some metrics. The values of interest forblocking are the netblock and the netblock size. A script extracts thenetblock and netblock size for each route in the table, and then eitherobtains an existing location or geolocates one IP network address in theblock by any of the existing methods and enter the result into the datawarehouse 30.

Using BGP routing tables: BGP routing tables have the same structure asinternal routing tables with minor exceptions. All routes in the BGPtable have a netblock size associated with them, and the route is givenin terms of AS paths. Most routes within a BGP table are of little usein determining a block because they do not take into account the routingperformed within an Autonomous System. However, BGP tables contain alarge number of exception routes. Very often, the blocks correspondingto these routes represent geographically compact domains, and thenetblock and netblock size can be used as extracted from the BGP table.Exception routes can be recognized easily since they are subsets ofother routes in the table. For example:

-   24.0.0.0/8 . . .-   24.32.0.0/24 . . .

The second route in the above example is a subset of the first route andis by definition an exception route.

(3) Using ISP topology maps: ISP topology maps usually contain thenetblocks that each router handles. These can be used as above. Theformat is nonstandard and requires decoding. A dedicated scripts createdeach topology map operates to parse these topology maps.

(4) Obtaining Internal Routing Tables: These tables can be obtained bystrategic alliances with ISPs. It is also possible obtain these bydialing up to an ISP account and running the same routing protocols asthe ISP network. This may convince the ISP routers that a dialog machineis also a router and the ISP routers may release internal routingtables.

(5) Obtaining BGP routing tables: Various sites on the web related toglobal routing release their copies of the global BGP routing table.

(6) Obtaining ISP topology maps: These can be obtained by alliances withan ISP.

The Unified Mapping Process (60)

The unified mapping process 61 operates to combine the results of anumber of mapping methodologies that do not yielded exact results (e.g.,combines the results of the inexact algorithms). In one embodiment, theunified mapping process 61 takes into account all information availablefrom such methodologies, and a probability (or confidence factor)associated with each, and establishes a unique location. The associatedprobability that serves as a confidence factor for the unique location.

In one embodiment, the unified mapping process 61 is implemented as aBayesian network that takes into account information regarding possiblecity and the state locations, results conflicts (e.g., there may becontradictory city/city indications or inconsistent cities/statecombinations, and calculates) a final unique location and the associatedprobability.

A probability for each of a number of possible locations that areinputted to the unified mapping process 61 is calculated utilizing theBayesian network, in one exemplary embodiment of the present invention.For example, if there is one possible location with a very highprobability and a number of other possible locations with smallerprobabilities, the location with the highest probability may be picked,and its associated probability returned. On the other hand, if they aremultiple possible locations with comparable probabilities, these may beforwarded for manual resolution, one embodiment of the presentinvention.

At a high level, the unified mapping process 61 receives a targetnetwork address (e.g., an IP address), and then runs the number ofnon-exact mapping processes as sub-tasks. These non-exact mappingprocesses then provide input to the Bayesian network. If one of thenon-exact algorithms fails, but a majority does not, the Bayesiannetwork will attempt to resolve the network address anyway.

FIG. 11 is a block diagram illustrating a process flow for the unifiedmapping process 61, according to an exemplary embodiment of the presentinvention. The unified mapping process 61 is an expert system suite ofalgorithms used to geolocate a network address (e.g., IP address). Theunified mapping process 61 combines a plethora of data from Internetregistries (Domain Name Server, Network IP Space, Autonomous SystemNumbers), Internet network connections (inferred via traceroutes), andworld geographical databases (place names, locations, populations). Theunified mapping process 61 further constructs a list of possiblephysical locations for a given network address, and from this list,through fuzzy logic and statistical methodologies, returns a locationwith a set of associated probabilities that provide an indicationregarding the accuracy of that location. In this way, the unifiedmapping process 61 can tie the network address to a specific geographiclocation (e.g. a city, country, zip/postal code, etc.) and provide anindication regarding the probability of the specific geographic locationbeing correct.

As shown in FIG. 11, the illustrated exemplary embodiment of the unifiedmapping process 61 has several components. Utilizing the data that havebeen gathered by external processes (e.g., the data collection agents18), a collection 120 of the location determination modules (LDMs)generate (1) location determinants (LDs) for a target address inquestion, and (2) and associated confidence factor (CF) or likelihoodthat the location determinant is correct (e.g., indicates a “true”geographic location). The location determinants generated by thecollection 120 of location determination modules are then passed througha location filter 122, which, based on certain criteria, removesnonsensical location determinants. After the filtering process performedby the location filter 122, location determinants and their associatedconfidence factors are passed into the location synthesis process (LSP)124, where the multitude of different (and similar) locationdeterminants, weighted by their confidence factors, compete against andcooperate with each other, ultimately yielding a unique and most likelylocation determinate including a “best estimate” geographic location(the location). Based on the degree of similarity between the “bestestimate” geographic location and its competing locations, differentconfidence factors are assigned for the geographic resolution levels,which are transformed by a confidence-accuracy translator (CAT) 126 intoa probability of accuracy for the winning location.

Confidence factors are used throughout the processing by the collection120 of location determination modules and are discussed in detail below.The confidence factors, in one embodiment present invention, come infour varieties (post-CM, post-LDM, post-LSP, and post-CAT), and theirmeanings are very different. The reader can use the context to determinewhich confidence factor is being referenced.

There are a number of data points that the unified mapping process 61utilizes. The specifics of how these are used are discussed below. Theseare also discussed above with respect to the data collection agents 18.

A location determination module (LDM) is a module that generates alocation determinant (LD) or set of location determinants that areassociated with the given network (e.g., IP) address. The locationdetermination modules utilize a variety of the available input data, andbased on the data's completeness, integrity, unequivocalness and degreeof assumption violation, assign a confidence factor for one or moregeographic locations. The location determination modules mayconceptually be thought of as experts in geolocation, each with a uniquespecial skills set. The location determination modules further makedecisions using “fuzzy logic”, and then present the output decisions(i.e., location determinants) and associated confidence factors (CFs) tothe location filter 122 and location synthesis process 124, where thelocation determinants are evaluated (or “argued”) democratically againstthe location determinants presented by other location determinationmodules.

All location determination modules operate it may somewhat similarmanner in that they each examine input data, and attempt to generatelocation determinants with an associated confidence factor based on theinput data. However, each location determination module is different inwhat input data it uses and how the respective confidence factors arederived. For instance, a specific location determination module mayextract location information from a hostname, while another analyzes thecontext of the traceroute; a further location determination module mayanalyze autonomous system information, while yet another makes use of aDNS Location record. By combining these distinct data inputs, eachindividually weighted by the parameters that most directly affect thelikelihood of the relevant data being correct, the location synthesisprocess 124 is equipped with a set of data to make a decision.

The location filter 122 operates through the location determinants,received from the collection 120 of location determination modules,which are in conflict with certain criteria. In particular, if ahostname ends with ‘.jp’, for example, the location filter 122 removesall location determinants that are not in Japan. Similarly, if ahostname ends with ‘.ca.us’, the location filter 122 omits locationdeterminants that are not in California, USA.

The location synthesis process 124, in one exemplary embodiment, isresponsible for the unification and congregation of all locationdeterminants that are generated by the collection 120 of locationdetermination modules. The location synthesis process 124 searches forsimilarities among the location determinants and builds a confirmationtable (or matrix that indicates correspondence (or agreement) betweenvarious location determinants. An intermediate result of this decisionmaking process by the location synthesis process 124 is the locationprobability table (LPT), an example of which is discussed above. Sincedeterminants may agree and disagree on multiple levels of geographicresolution (i.e. San Francisco, Calif. and Boulder, Colo. differ incity, state, and region, but are similar in country and continent), thelocation probability table develops different values at different levelsof geographic resolution. A combined confidence factor, which is alinear combination of each of the constituent confidence factor fields,is computed and used to identify a most likely location (the winninglocation) and an associated probability of the winning location beingcorrect.

The values contained in a location probability table, as returned fromthe location synthesis process 124, are translated by theconfidence-accuracy translator (CAT) 126 into a final form. A smallsubset of the data is run against verification data to compute therelationship between post-LSP confidence factors and accuracy. Giventhis relationship, the location probability table is translated toreflect the actual probability that the given network address wascorrectly located, thus completing the process of geolocation.

A discussion will now be presented regarding location determinationmodules, and fuzzy confidence maps, according to an exemplary embodimentpresent invention. This discussion provides an understanding of thelocation determination modules (LDMs) and their dominantdecision-facilitating mechanism, namely confidence maps (CMs).

A location determination module generates a location determinant (LD),or set of location determinants, and an associated confidence factor(CF), or set of confidence factors. These location determinants areprovided, together with an associated confidence factor, to the locationfilter 122 and onto the location synthesis process 124, where based onthe magnitude of their confidence factors and agreement with otherlocation determinants, are considered in the decision making of theunified mapping process 61. Eight exemplary location determinationmodules are discussed below. These exemplary location determinationmodules (LDMs) are listed below in Table 1, together with the source oftheir resultant location determinant, and are shown to be includedwithin the collection 120 of location determination modules shown inFIG. 11:

TABLE 1 List of exemplary LDMs LDM Name Source of LDM RegEx LDM 130String Pattern Matching in a Hostname. Net LDM 132 IP Registry DNS LDM134 Domain Name Server Registry ASN LDM 136 Autonomous System RegistryLoc LDM 138 DNS Loc Record LKH LDM 140 Last Known Host in a TracerouteNKH LDM 142 Next Known Host in a Traceroute Sandwich LDM 144 Combinationof LKH and NKH Suffix LDM 146 Last One or Two Words of Hostname

Further details regarding each of the above listed locationdetermination modules will be provided below, and an overview of twoexemplary location determination modules will be discussed as anintroduction.

The RegEx (Regular Expression) LDM 130, in an exemplary embodiment ofthe present invention, searches through a hostname and attempts toextract place names (cities, states, or countries) from within it. Thehost name may be obtained by performing a traceroute, or by issuing aNSLOOKUP or HOST command against a network address. Once the LDM 130identifies one or more place names, associated confidence factor values(based for example on parameters like city population, number of lettersin the string from which the name was extracted, distance to the lastknown host in a traceroute, etc.) are generated for each of the placenames.

The Net LDM 132 returns a geographic location for the network address(e.g., an IP address) as it is registered with the appropriate authority(e.g., ARIN/RIPE/APNIC). The confidence factor assigned to thegeographic location is based primarily on the size of the network blockthat is registered and within which the network address falls, under theassumption that a small network block (e.g., 256 or 512 hosts) can belocated in common geographic location, whereas a large network block(e.g., 65,536) is less likely to all be located in a common geographiclocation.

There are a number of advantages to utilizing confidence factorsthroughout the inner workings of the unified mapping process 61. By“fuzzifying” the data (e.g., treating every possible geographic locationas a viable answer with a confidence factor reflective of its dynamicaccuracy), then processing the data, and then “defuzzifying” (e.g.,collapsing onto one unique answer), the unified mapping process 61 isable to retain as much information as possible throughout the course ofprocessing data.

The formal translation of input data/parameters into LDM confidencefactors happens through relationships known as confidence maps (CMs).These relationships explicitly represent the correlation (orrelationship) between input parameters and the likelihood (orprobability) that an estimated geographic location for a network addressis in fact correct.

FIGS. 12A and 12B illustrate a one-dimensional confidence map 150 and atwo-dimensional confidence map 160 respectively, according to exemplaryembodiments of the present invention. Turning first to theone-dimensional confidence map 150, consider the exemplary scenario inwhich the Net LDM 132 returns a certain city. The question arises as tohow the Net LDM 132 can attach a level of certainty (or probability)that the city is a correct geolocation associated with a networkaddress. As stated above, in general, smaller network blocks are morelikely to yield a correct geographic location than large ones. Based onthis premise, a relationship between (1) the number of nodes within anetwork block and (2) a confidence level that a particular networkaddress is located in a city associated with that network block can bedetermined and expressed in a confidence map, such as the confidence map150 shown in FIG. 12A.

Interpreting FIG. 12A, it can be seen that confidence level for thegeographic location is very high if the network block size is small.However, as the size of the network block increases, the confidencelevel decreases. In an alternative embodiment of the present invention,as opposed using the “fuzzy logic” with confidence values, “crisp logic”may be utilized. The “crisp logic” implementation differs from the“fuzzy logic” implementation in that the “crisp logic” may implements apass/fail test. For example, rather crisp logic may specify thatnetworks smaller than size x are correctly located and larger than x areincorrectly located. On the other hand, the exemplary “fuzzy logic”implementation represented by the relationship shown in FIG. 12A presentthe continuum that represents a probabilistic relationship.

While one-dimensional confidence maps 150, such as that shown in FIG.12A, are good indicators of a likelihood of correct location, there aremany cases where a nonlinear interaction between two parameters makes atwo-dimensional confidence map 160, such as that shown in FIG. 12B, moreappropriate.

Consider the example where a RegEx LDM 130 extracts the strings ‘sf’ and‘santaclara’ out of a hostname. From these, consider that the RegEx LDM130 generates a number of possible geographic locations: San Francisco,Calif.; San Fernando, Calif.; Santa Fe, N.Mex.; South Fork, Colo.; andSanta Clara, Calif. With such ambiguity, the question arises as to howthe unified mapping process 61 may output an estimated geographiclocation. Again, by constructing an appropriate confidence map 160, suchas that shown in FIG. 12B, the unified mapping process 61 is enabled toseparate geographic locations of a high probability from those of a lowprobability. Specifically, the confidence map 160 relates (1) citypopulation (the y-axis) and (2) string length (the x-axis) to (3) aconfidence factor (color).

Interpreting the two-dimensional confidence map 160 shown in FIG. 12B,it will be noted that this confidence map 160 attributes a higherconfidence factor when the city is large and/or when the string fromwhich unified mapping process 61 extracted the location is long. Forexample, as ‘sf’ is a short string and subsequently prone to ambiguity,it does not have the same level of confidence that a long string such as‘santaclara’. However, if there is a large population associated with aspecific geographic location, then the weighting of the string length isdiscounted. For example, the two-dimensional confidence map 160, whenapplied to the aforementioned examples, yields the following Table 2:

TABLE 2 Example table of results from confidence map Location StringLength Population Confidence San Francisco, CA 2 700,000 35 SanFernando, CA 2  20,000 10 Santa Fe, NM 2  70,000 15 South Fork, CO 2 ?(<5,000) 0 Santa Clara, CA 10  90,000 40

Accordingly, through the use of a single confidence map such as thatshown in FIG. 12B, a location determination module (e.g., the Net LDM132) can separate reasonable location determinants from unreasonableones. However, as such separation may depend on a large number offactors, and the unified mapping process 61 may utilize a large numberof confidence maps.

In one embodiment, each location determination module uses a dedicatedset of confidence maps, and combines the results of each confidence map(for each location) by a weighted arithmetic mean. For example, ifcf_(i) is the i^(th) of n confidence factors generated by the i^(th) CM,with associated weight w_(i), then the combined confidence factor (CCF)is computed according to the following equation:${CCF} = \frac{\sum\limits_{i = 1}^{n}\quad{{cf}_{i}w_{i}}}{\sum\limits_{i = 1}^{n}\quad w_{i}}$

Every candidate geographic location must pass through each relevantconfidence map and has multiple confidence factors associated therewithcombined. Once a location determinant has a combined confidence factor,it no longer uses the multiple individual factors. Specifically, thelocation determinant and the associated combined confidence factor arecommunicated to the location filter 122 and subsequently the locationsynthesis process 124.

In the above examples, a confidence map may not assign a value higherthan 50 for confidence factor. Since the combined confidence factor isan average of these, it is also less than 50. If a confidence factor isgenerated by the location synthesis process to have a value greater than50, a confirming comparison may take place.

It should also be noted that a specific location determination modulemay utilize a mix of one-dimensional and two-dimensional confidencemaps, each of which has advantages and disadvantages. A one-dimensionalconfidence maps may lack the ability to treat multidimensional nonlinearinteraction, but only requires the one parameter to run. Conversely, atwo-dimensional confidence map can consider higher dimensionalinteraction effects, but if one of the parameters is missing, theconfidence map cannot be utilized to generate a confidence factor.

It should also be noted that the location determination modules aretruly modular, and that none depend on any other, and they can easily beadded, modified, or removed with respect to the unified mapping process61.

In one exemplary embodiment, as illustrated in FIG. 2, confidence maps33 are stored within the data collection database 26. The confidencemaps 33 are represented either as a matrix, or as a function where aninput parameter constitutes a continuum, as opposed to discrete values.To this end, FIG. 12C is an entity-relationship diagram illustratingfurther details regarding the storage of the confidence maps 33 withinthe data collection database 26. A reference table 35, which is accessedby an LDM, includes records that include pointers to a matrix table 37and a function table 39. The matrix table 37 stores matrices for thoseconfidence maps having input parameters that constitute discrete values.The function table 39 stores functions for those confidence maps forwhich an input parameter (or parameters) constitute a continuum.

RegEx (Regular Expression) LDM Location Generation

FIG. 13 is a flowchart illustrating a method 170, according to anexemplary embodiment of the present invention, performed by the RegExLDM 130 to identify one or more geographic locations for a networkaddress and to associated at least one confidence factor with each ofthe geographic locations. The RegEx LDM 130 performs a locationdetermination based on searching for string patterns within the hostname. Accordingly, the method 170 commences at block 172 with thereceipt of input data (e.g., a traceroute or other data collected by thedata collection agents 18). At decision block 174, a determination ismade as to whether one or more hostnames are included within the inputdata. If there is no hostname included within the input data (e.g., atraceroute) provided to the unified mapping process 61, the RegEx LDM130 exits at block 176.

On the other hand, if a hostname is included within input data, then theRegEx LDM 130 at block 178 parses the hostname by delimiter characters(e.g., hyphens, underscores, periods, and numeric characters) toidentify words that are potentially indicative of a geographic location.

At block 180, the RegEx LDM 130 runs comparisons on these newlyidentified words individually, and in conjunction with neighbor words,to check for similarity to patterns that correspond to geographiclocations (e.g., place names). In one embodiment, the RegEx LDM 130accesses the demographic/geographic database 31 contained within thedata warehouse 30 to obtain patterns to use in this comparisonoperation. In one embodiment, the LDM 130 checks individual words, anditeratively “chops” or removes letters from the beginning and end of theword in the event that extraneous characters are hiding valuableinformation. Strings that are more likely associated with networking andhardware than place names (such as ‘ppp’, ‘dsl’, ‘isdn’, ‘pop’, ‘host’,‘tel’, etc.) are not included in any pattern matching routines.

Examples of valid patterns, as stored within the demographic/geographicdatabase 31, that may be sought include various combinations of:

-   -   1. Full city name;    -   2. Full state name;    -   3. Full country name;    -   4. Two character abbreviation of city name (if and only if city        has a two part name);    -   5. Two character abbreviation of state name;    -   6. Two character abbreviation of country name;    -   7. Three character abbreviation of city name (if city has a        three part name);    -   8. First three characters of city name, including vowels;    -   9. First three characters of city name, excluding vowels;    -   10. First four characters of city name, including vowels;    -   11. First four characters of city name, excluding vowels;    -   12. Airport codes;    -   13. Common abbreviations for city names; and    -   14. Alternate spellings for city names.

The RegEx LDM 130 is capable of extracting fairly obfuscated geographicinformation from hostnames. One of the shortcomings, however, of thehistory of place naming is ambiguity. The RegEx LDM 130, at block 180,therefore accordingly generally identifies not one but many geographiclocations, and generates multiple location determinants.

The following table presents examples of the location determinants thatthe RegEx LDM 130 may generate from the exemplary host names:

TABLE 3 Example RegEx LDM location determinant construction ActualHostnames Location Determinants Rules/Reasons/Patternsdyn1-tnt4-1.chicago. Duyan, China Three character, no il.ameritech.netDayuan China vowel Deyang China Full city name Taunton, Massachusetts,Full state name USA Two character city Tonto Basin, Arizona, USA nameTanta, Egypt Two character country Taunton, Minnesota, USA codeTuntutuliak, Alaska, USA Tintah, Minnesota, USA Tontitown, Arkansas, USATontogany, Ohio, USA Chicago, Illinois, USA Illinois, USA Island Lake,Illinois, USA Indian Lake, New York, USA Israel p3-max50.syd. Sydney,Florida, USA Three character ihug.com.au Sydney, Australiac2501.suttonsbay. Sutton's Bay, Michigan, Full city name k12.mi.us USA(multiple words) pool-207-205-179- Phoenix, Maryland, USA Fourcharacter, no 101.phnx.grid.net Phoenix, New York, USA vowel Phoenix,Oregon, USA Phoenixville, Pennsylvania, USA Phoenix, Arizona, USAPhoenix, Virginia, USA resaleseattle1- Seattle, Washington, USA Fullcity name 1r7169.saturn.bbn. com usera723.uk. United Kingdom Twocharacter country uudial.com code

Through the usage of common abbreviations and alternate spellings, theRegEx LDM 130, for example, also knows to put ‘lsanca’ in Los Angeles,Calif., and ‘cologne’ in Köln, Germany.

Because of the large number of location determinants that the RegEx LDM130 can potentially generate, in one embodiment rules may restrictlocation determinant generation of trivially small (e.g., low populationor low connectivity index) cities from fewer than 4 characters.

The RegEx LDM 130 is particularly suited to identify geographiclocations associated with the Internet backbone/core routers. It is notuncommon for a company to make use of the hostname as a vehicle forcommunicating location. By using typical abbreviations and ageographical database of many tens of thousands of place names, theRegEx LDM 130 is suited to locating these hosts.

The RegEx LDM 130 has the ability to produce a multitude of locationdeterminants for a particular network address. Because the RegEx LDM 130is suited to identify geographic locations along the Internet backboneit may not, in one embodiment, be heavily deployed in the geolocation ofend node targets. Instead, the immediate (router) locations delivered bythe LDM 130 may be stored and used by other LDMs of the collection 120,which make use of these results as Last Known Hosts (LKHs) and NextKnown Hosts (NKHs).

Returning to the method 170 illustrated in FIG. 13, at block 180,multiple confidence maps are utilized to attach confidence factors tothe geographic locations identified and associated with a networkaddress at block 180. Further information regarding exemplary confidencemaps that may be used during this operation is provided below.

At block 184, the RegEx LDM 130 outputs the multiple geographic locationdeterminants, and the associated confidence factors, as a set to thelocation filter 122, for further processing. The method 170 then exitsat block 176.

Because of a degree of ambiguity and numerous location determinants thatmay be returned by the RegEx LDM 130, the LDM 130 employs a relativelylarge number of confidence maps when compared to other LDMs of thecollection 120. The confidence maps employed by the LDM 130, in oneexemplary embodiment, relate parameters such as word position, wordlength, city population, city connectivity, distance of city toneighboring hosts in the traceroute, etc.

An exemplary collection of confidence maps that may be utilized by theRegEx LDM 130 to attach confidence factors to location determinants isdiscussed below with reference to FIGS. 14A-14Q. It will be noted thateach of the confidence maps discussed below includes a “confidence mapweight”, which is a weighting assigned by the RegEx LDM 130 to aconfidence factor generated by a respective confidence map. Differentconfidence maps are assigned different weightings based on, inter alia,the certainty attached to the confidence factor generated thereby. Thenumber of terms or parameters of the confidence maps described belowrequire clarification. The term “hop ratio” is an indication of a hopposition within a traceroute relative to an end host (e.g., how far backfrom the end hosts a given hop is). The term “connectivity index” is ademographic representation of the magnitude or amount of network accessto which a location has access within a network. The term “minimumconnectivity” is a representation of a lowest common denominator ofconnectivity between to network entities (e.g., a Last Known Host and anend host). Distances between geographic locations are calculated once ageographic location has been determined. The latitude and longitudeco-ordinates of a geographic location may, in one exemplary embodiment,be utilized to performed distance calculations.

Hop Ratio—Connectivity Confidence Map (190)

-   X-axis: Hop Ratio (as determined from traceroute)-   Y-axis: Connectivity Index-   Color: confidence factor-   Confidence map weight: 40-   Comments: An exemplary embodiment of the confidence map 190 is    illustrated in FIG. 14A. This confidence map 190 is most assertive    in the middle of a traceroute where it provides well-connected    location determinants high confidence factors and less connected    location determinants low confidence factors. At the beginning and    the end of the traceroute, it has the opposite effect; well    connected location determinants receive lower confidence factors and    less connected get higher.

Word Length Confidence Map (190)

-   X-axis: Length of String-   Y-axis: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 192 is    illustrated in FIG. 14B. In place name string matching, a longer    string provides a high degree of certainty than a shorter string,    and decreases ambiguity. This confidence map 192 attributes higher    confidence factors for longer strings and confidence factors of zero    for two character strings.

Word Length—Number of Entries Confidence Map (194)

-   X-axis: Length of String-   Y-axis: Number of location determinants generated by the String-   Color: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 194 is    illustrated in FIG. 14C. The confidence map 194 couples the word    length (an indirect measure of ambiguity) with the number of    location determinants returned by the RegEx LDM 130 (a direct    measure of ambiguity). Strings that are too short and yield too many    location determinants are attributed a lower confidence factors than    unique ones. It will be noted that the confidence map 194 is    attributed a relatively higher weighting in view of the high degree    of certainty delivered by this confidence map 194.

Word Length—Population Confidence Map (196)

-   X-axis: Length of String-   Y-axis: Population-   Color: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 196 is    illustrated in FIG. 14D. As stated in the above, short words are    attributed relatively low confidence factors. Nonetheless, it is    desirable to attributed a relatively higher confidence factor to    geographic locations that are heavily populated, in spite of such    geographic locations being indicated by a short word. For example,    so that ‘sea’ and ‘sf’ (indicating Seattle and San Francisco,    respectively) are attributed higher confidence factors, this    confidence map 196 allows well-populated cities to be abbreviated    shortly.

Word Length—Connectivity Confidence Map (198)

-   X-axis: Length of String-   Y-axis: Connectivity Index-   Color: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 198 is    illustrated in FIG. 14E. For the same reasons discussed above with    reference to the confidence map 196 illustrated in FIG. 14D, well    connected cities are more likely to be correct than less connected    cities. The confidence map 198 seeks to ensure that even short    abbreviations are likely to be mapped correctly by attributing a    higher confidence factor too short words (e.g., abbreviations) that    exhibit a high degree of connectivity.

Distance to LKH—Hop Ratio of LKH Confidence Map (200)

-   X-axis: Distance in Miles to Last Known Host. This is determined    from the demographic/geographic database 31 that stores    intra-location distance values.-   Y-axis: Hop Ratio of Last Known Host-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 200 is    illustrated in FIG. 14F. Two hosts adjacent in a traceroute are    expected to be physically near each other, unless they are traversed    in the middle of the traceroute. This confidence map 200 is    reflective of this expectation. Hosts that are distant and at the    end of a traceroute are attributed lower confidence factors.

Distance to LKH—Node Distance to LKH Confidence Map (202)

-   X-axis: Distance in Miles to Last Known Host (LKH)-   Y-axis: Number of Hops Between this Host and LKH.-   Color: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 202 is    illustrated in FIG. 14G. Under the premise that a host should be    located near the last known host in a traceroute, the confidence map    202 gives lower confidence factors when the LKH is close in the    traceroute but far in physical space. The confidence map 202 is more    forgiving of hosts slightly further in the traceroute.

Distance to LKH—LKH Population Confidence Map (204)

-   X-axis: Distance in Miles to Last Known Host-   Y-axis: Minimum Population of this Host and LKH. This information is    again retrieved from the demographic/geographic database 31.-   Color: confidence factor-   Confidence map weight: 70-   Comments: An exemplary embodiment of the confidence map 204 is    illustrated in FIG. 14H. It is generally found that hops in a    traceroute jump great distances only when they travel from one major    backbone city to another. A common characteristic of these cities is    their large populations. So, in the confidence map 204, larger,    closer location determinants are rewarded, while distant, small ones    are punished.

Distance to LKH—LKH Connectivity Confidence Map (206)

-   X-axis: Distance in Miles to Last Known Host-   Y-axis: Minimum Connectivity of this Host and LKH-   Color: confidence factor-   Confidence map weight: 85-   Comments: An exemplary embodiment of the confidence map 206 is    illustrated in FIG. 14I. Similar to the preceding confidence map 204    based on population, this confidence map 206 rewards cities that are    generally well-connected. For example, cities like New York and    London can be connected to very distant cities.

Distance to NKH—Hop Ratio of NKH Confidence Map (208)

-   X-axis: Distance in Miles to Last Known Host-   Y-axis: Hop Ratio of Next Known Host-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 208 is    illustrated in FIG. 14J. Two hosts adjacent in a traceroute are    expected to be physically near each other, unless they are traversed    in the middle of the traceroute. The confidence map 208 is    reflective of this expectation. Hosts that are distant and at the    end of a traceroute receive lower confidence factors.

Distance to NKH—Node Distance to NKH Confidence Map (210)

-   X-axis: Distance in Miles to Next Known Host-   Y-axis: Number of Hops Between this Host and NKH-   Color: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 210 is    illustrated in FIG. 14K. Under the premise that a host should be    located near the last known host in a traceroute, the confidence map    210 attributes lower confidence factors when the NKH is close in the    traceroute, but far in physical space. The confidence map 210 is    more forgiving of hosts slightly further in the traceroute.

Distance to NKH—NKH Population Confidence Map (212)

-   X-axis: Distance in Miles to Next Known Host-   Y-axis: Minimum Population of this Host and NKH-   Color: confidence factor-   Confidence map weight: 70-   Comments: An exemplary embodiment of the confidence map 212 is    illustrated in FIG. 14L. Hops in a traceroute tend to jump great    distances only when they travel from one major backbone city to    another. A common characteristic of these backbone cities is their    large populations. Accordingly, the confidence map 212 generates a    confidence factor such that larger, closer location determinants are    rewarded, while distant, small location determinants are punished.

Distance to NKH—NKH Connectivity Confidence Map (214)

-   X-axis: Distance in Miles to Next Known Host-   Y-axis: Minimum Connectivity of this Host and NKH-   Color: confidence factor-   Confidence map weight: 85-   Comments: An exemplary embodiment of the confidence map 214 is    illustrated in FIG. 14M. The confidence map 214 rewards cities that    are generally well-connected. For example, cities like New York and    London can be connected to very distant cities.

Population Confidence Map (216)

-   X-axis: Population-   Y-axis: confidence factor-   Confidence map weight: 40-   Comments: An exemplary embodiment of the confidence map 216 is    illustrated in FIG. 14N. Generally speaking, the population of a    geographic location is an effective measure of likelihood.    Intuitively, the Moscow of the Russian Federation is more likely    than the Moscow of Iowa. Especially in the USA, population may be a    powerful indicator of the likelihood of location determinant    correctness.

Neighboring Connectivity Confidence Map (218)

-   X-axis: Mean of LKH and NKH Connectivity Indices-   Y-axis: Connectivity Index-   Color: confidence factor-   Confidence map weight: 90-   Comments: An exemplary embodiment of the confidence map 218 is    illustrated in FIG. 14O. A base premise of the confidence map 218 is    that connectivity indices along a traceroute ought to be continuous.    That is: host locales go from low connectivity to medium, to high.    Any host's connectivity index along a traceroute ought theoretically    not to deviate from the mean of its neighbors. This map penalizes    such a deviation.

Connectivity Confidence Map (220)

-   X-axis: Connectivity Index-   Y-axis: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 220 is    illustrated in FIG. 14P. The connectivity index is utilized by the    confidence map 220 to provide a direct measure of the probability    that a host is in the particular geographic location. According to    the confidence map 220, the better connected a geographic location    (e.g., city) is, the more likely the host is to be at a geographic    location.

Word Position Confidence Map (222)

-   X-axis: Position of 1^(st) Character of Word in Hostname-   Y-axis: confidence factor-   Confidence map weight: 20-   Comments: An exemplary embodiment of the confidence map 222 is    illustrated in FIG. 14Q. It will be noted that the confidence map    222 is assigned a relatively low confidence map weight, which is    indicative of a relatively low effectiveness of the confidence map    222. It has been found that information in a hostname is more likely    to be found at the extreme ends than in the middle. Also if two city    names appear together in a hostname, the names toward the ends of    the word tend to have more relevance.

Network (Net) LDM Location Generation

FIG. 15 is a flowchart illustrating a method 240, according to anexemplary embodiment of the present invention, performed by the Net LDM132 to identify one or more geographic locations for a network address(or block of network addresses) and associate at least one confidencefactor with each of the geographic locations.

At block 242, the Net LDM 132 initiates external data collectionroutines (e.g., data collection agents 18) to query multiple InternetProtocol (IP) registering authorities (e.g., RIPE/APNIC/ARIN) to asmallest possible network size

At block 244, geographical information (e.g., city, state, country, thezip/postal code, area code, telephone prefix) is parsed from the queryresults and extracted and stored along with the network address range atblock 246.

At block 248, the Net LDM 132 utilizes multiple confidence maps toattach confidence factors to each of the geographic locations identifiedat block 244, or to each of the geographic information items identifiedat block 244.

At block 250, the Net LDM 132 outputs the multiple geographic locations(or geographic information items) and the associated confidence factorsto the location filter 122. The method 240 then terminates at block 252.

Because the Net LDM 132 may be of limited effectiveness along the corerouters, the use of the Net LDM 132 may, in one exemplary embodiment, berestricted to the last three hops of a traceroute. The Net LDM 132 mayoptionally also not be utilized if a network block size registered islarger than 65,536 hosts, for it is unlikely that so many machines wouldbe located in the same place by the same organization.

The Net LDM 132 is a particularly effective at generating accurateconfidence factors for geographic locations when the network blocksregistered with the IP registering authority are relatively small (e.g.,less than 1024 hosts). If the Net LDM 132 incorrectly attached is a highconfidence level to a geographic location, it is most likely related toa large network block or an obsolete record in a registry.

The confidence factors generated by the Net LDM 132 come from distanceto a Last Known Host (LKH) and a Next Known Host (NKH) (e.g., calculatedutilized in the latitude and longitude co-ordinates of these hosts) thesize of the network block, a position in a traceroute (e.g., relativelocation near the end of the traceroute), population and connectivity.Regarding position within a traceroute, it will be appreciated that arelative position within the traceroute will be dependent upon thenumber of hops, and the relevant hop's position within that number ofhops. For example, if they are 7 hops within a given traceroute, thenhop 6 is considered to be near the end host. However, if there are 20hops within the traceroute, hop 6 to be considered to be very distantfrom the end host.

An exemplary collection of confidence maps that may be utilized by theNet LDM 132 to attach confidence factors to location determinants arediscussed below with reference to FIGS. 16A-16E. It will be noted fromthe following discussion of the confidence maps utilized by the Net LDM132 that, while distance and hop ratio are used in similar ways as inthe RegEx LDM 130, population and connectivity are used in contraryways. Again, different confidence maps are assigned different weightingsbased on, inter alia, the certainty attached to the confidence factorsgenerated thereby.

LKH Distance—Hop Ratio Confidence Map (260)

-   X-axis: Distance in Miles Between LKH and Net-   Y-axis: Hop Ratio-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 260 is    illustrated in FIG. 16A. The confidence map 260 generates a    relatively high confidence factor only at the ends of a traceroute    and only when a geographic location (e.g., a city) corresponding to    the network addresses within close proximity to the LKH.

Net Size Confidence Map (262)

-   X-axis: Number of Nodes in Registered Block-   Y-axis: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 262 is    illustrated in FIG. 16B. The confidence map 262 works off of two    premises. First, if an entity has gone through the trouble to    register a small block of network space, it is probably accurate.    Conversely, large networks that are registered to one organization    probably have the hosts spread out across a large area. Thus, the    confidence map 262 operates such that small network sizes yield    large confidence factors.

NKH Distance—Hop Ratio Confidence Map (264)

-   X-axis: Distance in Miles Between LKH and Net-   Y-axis: Hop Ratio-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 264 is    illustrated in FIG. 16C. The confidence map 264 generates a    relatively high confidence factor for a geographic location only at    the ends of a traceroute and only when a geographic location (e.g.,    a city) corresponding to network addresses within close proximity to    the NKH.

Connectivity Confidence Map (266)

-   X-axis: Connectivity Index-   Y-axis: confidence factor-   Confidence map weight: 25-   Comments: An exemplary embodiment of the confidence map 266 is shown    in FIG. 16D. Contrary to the relationship in the RegEx LDM 130, here    less-connected geographic locations (e.g., cities) are rewarded with    higher confidence factors. The premise is that if a network is    registered in a small town, hosts on that network are more likely to    be in that small town. Larger cities may just be corporate    headquarters.

Population Confidence Map (268)

-   X-axis: Population-   Y-axis: confidence factor-   Confidence map weight: 25-   Comments: An exemplary embodiment of the confidence map 268 is    illustrated in FIG. 16E. Contrary to the relationship in the RegEx    LDM 130, here smaller geographic locations are rewarded with higher    confidence factors. The premise is that if a network is registered,    for example, in a small town, hosts on that network are more likely    to be in that small town. Larger cities may just be corporate    headquarters.

Domain Name Server (DNS) LDM Location Generation

FIG. 17 is a flowchart illustrating a method 270, according to anexemplary embodiment of the present invention, performed by the DNS LDM134 to identify one or more geographic locations for a network address(or block of network addresses) and to associate at least one confidencefactor with each of the geographic locations.

At block 272, the DNS LDM 134 initiates external data collectionroutines (e.g., data collection agents 18) to query multiple Domain NameServer (DNS) registering authorities to collect DNS records. Theserecords correspond to ownership of a particular domain name (e.g.,www.harvard.com or www.amazon.com)

At block 274, geographical information (e.g., city, state, country, thezip/postal code, area code, telephone prefix) is parsed from the DNSrecords and extracted and stored along with the domain name at block276.

At block 278, the DNS LDM 134 utilizes multiple confidence maps toattach confidence factors to each of the geographic locations identifiedat block 274.

At block 280, the DNS LDM 134 outputs the multiple geographic locations(or geographic information items) and the associated confidence factorsto the location filter 122. The method 270 then terminates at block 282.

Similar to the Net LDM 132, the DNS LDM 134 may not be most effectivealong the backbone core routers. For example, it is not helpful to knowthat att.net is in Fairfax or that exodus.net is in Santa Clara. Toavoid potential problems related to this issue, the DNS LDM 134 may bedeployed only on the last three hops of a traceroute, in one exemplaryembodiment of the present invention.

If a DNS record, retrieved at block 272 indicates the same geographiclocation as a network record, retrieved at block 242, then it may beassumed, in one exemplary embodiment, that this geographic location is acorporate office and that the actual hosts may or may not be at thatlocation. To prevent the location synthesis process 124 from beingoverwhelmed by redundant data that might not be useful, the DNS LDM 134is prevented from duplicating the Net LDM 132, because, in an exemplaryembodiment, the LDM 134 is less skillful than the LDM 132.

Similar to the Net LDM 132, the DNS LDM 134 may be strongest at the endof a traceroute, but not along the backbone core routers. Accordingly,the DNS LDM 134 may work well to geolocate companies that have a domainname registered and do their own hosting locally. Small dial-up ISPs arealso locatable in this way as well.

An exemplary collection of confidence maps that may be utilized by theDNS LDM 134 to attach confidence factors to location determinants, atblock 278, are discussed below with reference to FIGS. 18A-18E. The DNSLDM 134 relies on similar parameters as the Net LDM 132 for determiningits confidence factors. Major differences include using distance to anetwork location, the rather than a network block size. It will also benoted that, in the exemplary embodiment, DNS confidence factors yieldedby the confidence maps discussed below are significantly lower than inother LDMs.

LKH Distance—Hop Ratio Confidence Map (290)

-   X-axis: Distance in Miles Between LKH and DNS-   Y-axis: Hop Ratio color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 290 is    illustrated in FIG. 18A. This confidence map 290 generates a    relatively high confidence factor only at the ends of a traceroute    and only when the geographic location (e.g., a city) corresponding    to the DNS record is within close proximity to the LKH.

Distance to Net Confidence Map (292)

-   X-axis: Distance in Miles Between Net and DNS-   Y-axis confidence factor-   Confidence map weight: 80-   Comments: An exemplary embodiment of the confidence map 292 is    illustrated in FIG. 18B. This confidence map 292 works under the    assumption that if the Net and DNS records are identical, then they    probably point to a corporate headquarters. If the distance between    the two is zero, then the confidence factor is zero. If, however,    the distance is not zero but is very small, then there is a greater    chance that either one could be correct, or a larger confidence    factor is given.

NKH Distance—Hop Ratio Confidence Map (294)

-   X-axis: Distance in Miles Between NKH and DNS-   Y-axis: Hop Ratio color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of this confidence map 294 is    illustrated in FIG. 18C. This confidence map 294 gives high    confidence only at the ends of a traceroute and only when the    geographic location (e.g., the city) corresponding to the DNS record    is within close proximity to the NKH.

Connectivity Confidence Map (296)

-   X-axis: Connectivity Index-   Y-axis: confidence factor-   Confidence map weight: 25-   Comments: An exemplary embodiment of this confidence map 296 is    illustrated in FIG. 18D. Contrary to the relationship in the RegEx    LDM 130, the DNS LDM 134 operates such that less-connected    geographic locations (e.g., cities) are rewarded with higher    confidence factors. The premise is that, for example, if a domain    name is registered in a small town, hosts associated with it are    more likely to be in that small town. Larger cities may just be    corporate headquarters or collocations.

Population Confidence Map (298)

-   X-axis: Population-   Y-axis: confidence factor-   Confidence map weight: 25-   Comments: An exemplary embodiment of the confidence map 298 is    illustrated in FIG. 18E. Contrary to the relationship in the RegEx    LDM 130, here smaller geographic locations (e.g., small towns) are    rewarded with higher confidence factors. The premise is that, for    example, if a domain name is registered in a small town, hosts    associated with it are more likely to be in that small town. Larger    cities may just be corporate headquarters.

ASN LDM Location Generation

The method by which the Autonomous System Network (ASN) LDM 136 operatesto identify one more geographic locations for network addresses, and toassign at least one confidence factor to each of the geographiclocations, is similar to the methods 240 and 270 of other two internetregistry LDMs (i.e., the Net LDM 132 and the DNS LDM 134). Specifically,as opposed to the deploying external data collection routines to gatherNet and DNS records, the ASN LDM 136 deploys the external datacollection routines to gather the Autonomous System data, and parse itfor meaningful geographic data. If ASN data is available, then the ASNLDM 136 can run.

The ASN LDM 136 is, in one embodiment, not used if the network blocksize registered by a blocking algorithm is larger than 65,536 hosts, asit is unlikely that so many machines would be located at a commonlocation under the same Autonomous System (AS).

As with the DNS LDM 134, the ASN LDM 136 does not run if its ASN recordmatches that of the Net LDM. Again, this is to avoid erroneousduplication.

The ASN LDM 136 is reliable because the ASN data is utilized in realnetwork communication, and is accordingly generally current, correct,and of a reasonable high resolution.

An exemplary collection of confidence maps that may be utilized by theASN LDM 136 to attach confidence factors to location determinants arediscussed below with reference to FIGS. 19A-19E. The confidence factorsgenerated by the ASN LDM 136 come from distance to LKH and NKH, the sizeof the network, the position in the traceroute, population andconnectivity. It will be noted that the following confidence maps, whileutilizing distance and hop ratio in similar ways as in the RegEx LDM130, population and connectivity are used in contrary ways.

LKH Distance—Hop Ratio Confidence Map (300)

-   X-axis: Distance in Miles Between LKH and ASN-   Y-axis: Hop Ratio color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 300 is    illustrated in FIG. 19A. This confidence map 300 gives high    confidence only at the ends of a traceroute and only when the    geographic location (e.g., a city) corresponding to the ASN record    is within close proximity to the LKH.

Net Size Confidence Map (302)

-   X-axis: Number of Nodes in AS Block-   Y-axis: confidence factor-   Confidence map weight: 100-   Comments: An exemplary embodiment of the confidence map 302 is    illustrated in FIG. 19B. This confidence map 302 operates off of two    premises. First, if an entity has gone through the trouble to    register a small block of network space, it is probably accurate.    Conversely, large networks that are registered to one organization    probably have the hosts spread out across a large area. Thus, small    net sizes yield large confidence factors.

NKH Distance—Hop Ratio Confidence Map (304)

-   X-axis: Distance in Miles Between LKH and ASN-   Y-axis: Hop Ratio-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 304 is    illustrated in FIG. 19C. This confidence map 304 generates    relatively high confidence factors only at the ends of a traceroute    and only when the geographic location (e.g., city) corresponding to    the ASN record is within close proximity to the NKH.

Connectivity Confidence Map (306)

-   X-axis: Connectivity Index-   Y-axis: confidence factor-   Confidence map weight: 25-   Comments: An exemplary embodiment of the confidence map 306 is    illustrated in FIG. 19D. Contrary to the relationship in the RegEx    LDM 130, here less-connected geographic locations (e.g., cities) are    rewarded with higher confidence factors. The premise is that if a    network is registered in a relatively smaller geographic location    (e.g., small town), hosts on that network are most likely in that    smaller geographic location. Larger cities may be corporate    headquarters.

Population Confidence Maps (308)

-   X-axis: Population-   Y-axis: confidence factor-   Confidence map weight: 25-   Comments: An exemplary embodiment of the confidence map 308 is    illustrated in FIG. 19E. Contrary to the relationship in the RegEx    LDM 130, here smaller geographic locations (e.g., smaller cities)    are rewarded with higher confidence factors. The premise is that if    a network is registered in, for example, a small town, hosts on that    network are most likely to be located in that small town. Larger    cities may be corporate headquarters.

Location (Loc) LDM Location Generation

The method by which the Loc LDM 138 operates to identify one moregeographic locations for network address, and to associate least oneconfidence level with each of the geographic locations, is again similarto the methods 240 and 270 of the Net and DNS LDMs 132 and 134 in thatexternal collection processes gather Location (Loc) records fromappropriate registries, which are parsed to extract locationdeterminants. The Loc LDM 138 differs from the above described LDMs inthat a collection of confidence maps is not utilized to attachconfidence factors to each of these location determinants, as will bedescribed in further detailed below.

The Loc LDM 138, in one exemplary embodiment, differs from thepreviously described LDMs in that it exhibits a high degree of accuracyand precision. Specifically, a DNS Loc record, as collected by externalprocesses, may provide an indication of a hosts' latitude and longitudedata, which may be utilized to tie a location determinant to a city (oreven smaller).

DNS Loc records are rarely available. Fewer than 1% of all hostsactually have a Loc record available.

The Loc LDM 138 is one of only two LDMs that do not make use ofconfidence maps. The rationale behind this is that there are nocircumstances that would change the belief in the highly accurate DNSLoc record, used by the Loc LDM 138. So as opposed to utilizing a numberof confidence maps, if the Loc record is available, the Loc LDM 138communicates a location determinant derived from the Loc record to thelocation filter 22, accompanied by a precise confidence factor, forexample, 85.

LKH LDM Location Generation

The LKH LDM 140 makes use of traceroute contextual data, and assertsthat the host in question is in precisely the same location as the onepreviously identified in the traceroute. Specifically, it is generallyfound that at the end of a traceroute, the physical distance from theone hop to the next is on the order of miles, not hundreds of miles. Itis also not uncommon for a traceroute to spend several hops in the samearea (i.e. network center).

Take, for instance, a partial traceroute to www.quova.com:

-   1 <10 ms <10 ms <10 ms 10.0.0.1-   2 30 ms 20 ms 21 ms loop1.dnvr-6400-gw1.dnvr.uswest.net    [63.225.108.254]-   3 270 ms 20 ms 30 ms 103.port1.dnvr-agw2.dnvr.uswest.net    [207.225.101.126]-   4 20 ms 20 ms 20 ms gig3-0.dnvr-gw2.dnvr.uswest.net    [206.196.128.219]-   5 20 ms 20 ms 20 ms h4-0.denver-cr2.bbnplanet.net [4.0.212.245]-   6 50 ms 20 ms 20 ms p4-0-0.denver-br2.bbnplanet.net [4.0.52.21]-   7 30 ms 30 ms 20 ms p0-0-0.denver-br1.bbnplanet.net [4.0.52.17]-   8 50 ms 60 ms 50 ms p2-3.lsanca1-ba2.bbnplanet.net [4.24.6.1]-   9 50 ms 60 ms 50 ms p7-0.lsanca1-br2.bbnplanet.net [4.24.4.38]-   10 50 ms 51 ms 60 ms p2-0.lsanca1-br1.bbnplanet.net [4.24.4.13]-   11 70 ms 70 ms 60 ms p7-3.paloalto-nbr2.bbnplanet.net [4.24.5.210]-   12 70 ms 60 ms 70 ms p1-0.paloalto-cr2.bbnplanet.net [4.0.6.78]-   13 2624 ms 2654 ms * pos2-1.core1.Sanjose1.Level3.net [209.0.227.1]-   14 230 ms 220 ms 221 ms so-4-0-0.mp2.Sanjose1.level3.net    [209.247.11.9]-   15 120 ms 130 ms 121 ms loopback0.hsipaccess1.Washington1.Level3.net    [209.244.2.146]-   16 280 ms 131 ms 130 ms 209.244.200.50

It will be noted that three consecutive hops (1-3) are all in Denverunder uswset.net, and the three following that are also in Denver underbbnplanet.net. In three following hops are all in Los Angeles. While theabove exemplary traceroute could be interpreted, in one embodiment,solely within the RegEx LDM 130, the LKH LDM 140 may operate toreinforce the results that the RegEx LDM 130 generates. This interactionis discussed in further detailed below.

While the LKH LDM 140 may provide useful results, it has with it adangerous side effect that requires careful attention; unless kept incheck, the LKH LDM 140 has the power to “smear” a single location overthe entire traceroute. The confidence maps utilized by the LDM 140, asdescribed below, are particularly strict to address this issue.

An exemplary collection of confidence maps that may be utilized by theLDM 140 to attach confidence factors to location determinants arediscussed below with reference to the FIGS. 20A-20C.

The below discussed collection of confidence maps attempt to address thefollowing issues relating to confidence factors associated with alocation determinant outputted by the LDM 140:

(1) How many nodes back was the last known host? If it was only one, itis probably a reasonable location determinant and deserves a highconfidence factor.

(2) Did the last known host have a high confidence factor? If it didnot, then neither should this one.

(3) Where in the traceroute is the last known host? If it is toward themiddle, then the two machines are less likely to be in the same placethan if it is at the end.

(4) Is the last known host physically located near to any of the Net,Loc, or DNS records for the host in question? If so, there is a higherlikelihood that the two are in the same place.

The below discussed collection of confidence maps parameterizes theabove concerns, generating confidence factors for the LKH LDM 140.

Node Distance—Confidence Confidence Map (320)

-   X-axis: Number of Hops Between this Host and the LKH-   Y-axis: Stored confidence factor of the LKH-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 320 is    illustrated in FIG. 20A . As such above, it is desirable that the    confidence maps utilized by the LDM 140 are “strict” to avoid    erroneous location determinant smearing. This confidence map 320    only attributes relatively high confidence factors if the LKH is a    small number of hops (e.g., less than 2 hops) away and the    confidence factor of the LKH is very high.

Node Distance—Hop Ratio Confidence Map (322)

-   X-axis: Number of Hops Between current Host and the LKH-   Y-Axis: Hop Ratio-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 322 is    illustrated in FIG. 20B. This confidence map 322 generates    relatively high factors if and only if the hosts are close together    (in the traceroute) and at the end of the traceroute. Other    scenarios receive low or zero confidence factors.

Shortest Registry Distance Confidence Map (324)

-   x-axis: Shortest Distance in Miles to {Net,DNS,Loc}-   y-axis: confidence factor-   confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 324 is    illustrated in FIG. 20C. The confidence map 324 gives slightly    higher confidence factors if and only if the LKH is proximal to any    of the Net, DNS, or Loc Records.

NKH LDM Location Generation

The mechanics of Last Known Host (LKH) LDM 140 are substantially similarto the Next Known Host (NKH) LDM 142. While the NKH will usually not bedirectly instrumental in geolocating an end node, it can play anauxiliary role, and provide useful supplemental information. Forexample, if Router A is the last hop before a traceroute goes to an endnode in, say, Denver, Colo., then it is not unlikely that Router A isalso in Denver, Colo. By assigning Router A to Denver, Colo., the nexttime a traceroute runs through Router A, it can use the LKH to press onfurther.

The NKH LDM 142, in a slightly less robust way than the LKH LDM 140 andin a substantially way than the RegEx LDM 130, is a mechanism forproviding supplemental information in the router space of the Internet,which subsequently provides aid in the end node geolocation.

An exemplary collection of confidence maps that may be utilized by theNKH LDM 142 to attach confidence factors to location determinants arediscussed below with reference to FIGS. 21A-21C.

Node Distance—Confidence Confidence Map (330)

-   X-axis: Number of Hops Between this Host and the NKH-   Y-axis: Stored confidence factor of the NKH-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 330 is    illustrated in FIG. 21A. Again it is desirable that the confidence    maps utilized by the NKH LDM 142 are “strict” to avoid erroneous    location determinant smearing. This confidence map 330 only gives    high confidence factors if the NKH is a small number of hops (e.g.,    less than 2 hops) away from a current geographic location (e.g.,    host) and the confidence factor of the NKH is very high.

Node Distance—Hop Ratio Confidence Map (332)

-   X-axis: Number of Hops Between current Host and the NKH-   Y-axis: Hop Ratio-   Color: confidence factor-   Confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 332 is    illustrated in FIG. 21B. This confidence map 332 gives relatively    high confidence factors if and only if the hosts are close together    (in the traceroute) and at the end of the traceroute. Other    scenarios receive low or zero confidence factors.

Shortest Registry Distance Confidence Map (334)

-   x-axis: Shortest Distance in Miles to {Net,DNS,Loc}-   y-axis: confidence factor-   confidence map weight: 50-   Comments: An exemplary embodiment of the confidence map 334 is    illustrated in FIG. 21C. The confidence map 334 gives slightly    higher confidence factors if and only if the NKH is proximal to any    of the Net, DNS, or Loc Records.

Sandwich LDM Location Generation

FIG. 22 is a flowchart illustrating a method 340, according to anexemplary embodiment of the present invention, performed by the sandwichLDM 144 to identify one more geographic locations for a network address,and associated at least one confidence factor with each of thegeographic locations.

The method 340 commences at decision block 342, where the sandwich LDM144 determines whether both the LKH and the NKH LDMs 140 and 142generated respective location determinants and associated confidencefactors. If not, and only one or neither of these LDMs 140 and 142generated a location determinant, the method 340 then ends at block 352.

On the other hand, following a positive determination at decision block342, at block 344 the sandwich LDM 144 retrieves the respective locationdeterminants from the LKH and the NKH LDMs 140 and 142.

At block 346, the sandwich LDM 144 identifies the location determinantreceived at block 344 that has the highest confidence factor associatedtherewith.

At block 348, the sandwich LDM 144 assigns a confidence factor to thelocation determinant identified at block 346 based on: (1) a combinationof the confidence factors assigned to each of the location determinantsby the LDMs 140 and 142 (e.g., by calculating the mean of the locationdeterminants); and (2) the distance between the location determinantsgenerated by the LDMs 140 and 142.

At block 350, the identified location determinant, and the newconfidence factor calculated at block 348 are outputted from thesandwich LDM 144 to the location filter 122. The method 340 then ends atblock 352.

It will be noted that the sandwich LDM 144 is different from the otherLDMs, because it is the only LDM that does not operate to produce alocation determinant that is potentially distinct from the locationdeterminants produced by the other LDMs. The sandwich LDM 144 works asan extra enforcer to further empower the LKH and NKH LDMs 140 and 142.For example, if an exemplary host has a LKH location determinant and aNKH location determinant, the sandwich LDM 144 will choose the moreconfident of the two location determinants and assign a confidencefactor based on their joint confidence factors and their distance to oneanother.

The sandwich LDM 144 addresses a potential inability of LKH and NKH LDMs140 and 142 to work together successfully in filling in so-called “surething” gaps. For example, if hop #10 of a traceroute is in New York Cityand hop #13 is in New York City, then it can be assumed with a highdegree of certainty that hops #11 and #12 should also be in New YorkCity. This scenario is then generalized to treat not just identical NKHand LKH location determinants, but also ones that are very close to oneanother.

The sandwich LDM 144, in an exemplary embodiment, utilizes a singleconfidence map 354 illustrated in FIG. 23 to assign a confidence factorto a location determinant.

Sandwich/Confidence Factor—Proximity Confidence Map (354)

-   X-axis: Distance in Miles Between LKH and NKH-   Y-axis: Mean confidence factor of LKH and NKH location determinants-   Color: confidence factor-   Confidence map weight: 50-   Comments: After the sandwich LDM 144 identifies which of the NKH or    LKH location determinants as a higher confidence factor, it assigns    a confidence factor to the identified location determinant that is    only nontrivial if the LKH and NKH location determinants are very    close and have a high mean confidence factor.

Suffix LDM Location Generation

The suffix LDM 146 operates on hostnames. If a hostname is notavailable, the suffix LDM 146 does not run. Further, it requires thatthe hostname end in special words, specifically ISO country codes orstate/province codes. Accordingly, the suffix LDM 146 does not employartificial intelligence, and looks up the code (e.g., the ISO countrycode or a state/province code) and returns the corresponding geographiclocation information. The code lookup may be performed on thedemographic/geographic database 31. For example, a hostname that ends in‘.jp’ is assigned to Japan; a hostname that ends in ‘.co.us’ is assignedto Colorado, USA.

In addition to the country and state standards, the suffix LDM 146 canalso identify dozens of large carriers that have presences in particularregions. For example, a hostname that ends in ‘.telstra.net’ is assignedto Australia; a hostname that ends in ‘.mich.net’ is assigned toMichigan, USA.

The suffix LDM 146 also has a special relationship with the locationfilter 122. Because of its accuracy and generally large scale, thesuffix LDM 146 is the only LDM that can insert location determinantsinto the location filter 122, requiring that all other locationdeterminants agree with the location determinant generated by the suffixLDM 146, or they are not permitted to pass onto the location synthesisprocess 124.

Similar to the Loc LDM 138, the likelihood of accuracy of the locationdeterminant generated by the suffix LDM 146 is not considered to becircumstantial. Accordingly, the suffix LDM 146 attributes a staticconfidence factor for all location determinants that it returns. Thisstatic confidence factor may, for example, be 91.

Location Filter (122)

In general, the spectrum of LDM “intelligence” is fairly large and, aswill be appreciated from the above description, ranges from thethorough, hard-working RegEx LDM 130, which may attempt to put ahostname with ‘telco’ in Telluride, Colo., to the precise Loc LDM 138,which may generate precise location determinants. While the locationsynthesis process 124, as will be described in further detail below, isintelligent enough to process a broader range of location determinantsutilizing corresponding confidence factors, it is desirable to removeunreasonable location determinants from the location determinants thatare forwarded to the location synthesis process 124 for consideration.

To this end, the suffix LDM 146, for example, has a very high successrate in geolocation of a plethora of hosts, especially foreign ones.While the suffix LDM 146 lacks the high precision to be used by itself,the location determinant produced thereby may, in one exemplaryembodiment, be deployed as a “filter location determinant”. Such afilter location determinant may, for example, be utilized by thelocation filter 122 to remove from the unified mapping process 61location determinants that do not show a predetermined degree ofcorrelation, agreement or consistency with the filter locationdeterminant. A filter location determinant may, for example, be deployedto remove noise data, retaining a smaller, more manageable subset oflocation determinants that can be processed more quickly by the locationsynthesis process 124.

In one exemplary embodiment, the location filter 122 is tied directly tothe suffix LDM 146. Because of the reliability and accuracy of thesuffix LDM 146, the location determinant produced by this LDM 146 may bedesignated as the “filter location determinant”.

FIG. 24 is a flowchart illustrating a method 360, according to anexemplary embodiment of the present invention, of filtering locationdeterminants received from the collection of LDMs utilizing a filterlocation determinant.

The method 360 commences at block 362 with the running of a highaccuracy LDM (e.g., the suffix LDM 146) to generate the “filter locationdeterminant” and optionally an associated confidence factor. At block364, after the suffix LDM 146 has executed, the filter locationdeterminant and confidence factor generated thereby are communicated tothe location filter 122.

At block 366, the location filter 122 determines whether the receivedfilter location determinant is a state or country. At block 368, thelocation filter 122 intercepts multiple location determinants outputtedby the collection of LDMs and bound for the location synthesis process124. The location filter 122 then checks to see if each of theselocation determinants adequately agrees with the filter locationdeterminant. If they do, at block 372, the location determinants proceedonward to the location synthesis process 124 by being retained in aninput stack being for this process 124. If they do not, at block 374,then the location determinants are removed from the input stack for thelocation synthesis process 124.

The agreement between the filter location determinant, and anyone of themultiple other location determinants received from the collection ofLDMs, in one exemplary embodiment of the present invention, is aconsistency between a larger geographic location (i.e., a locationdeterminant of a relatively lower geographic location resolution)indicated by the filter location determinant and a more specificgeographic location (i.e., a location determinant of a relatively highergeographic location resolution) that may be indicated by a subjectlocation determinant. For example, location filter 122 may be effectivein the debiasing of the United States data set. If the word ‘london’ isextracted from a hostname by way of the RegEx LDM 130, then the locationsynthesis process 124 may have a dozen or so ‘Londons’ to sort out. Oneis in the UK, and all the others are in the US. The confidence factorsgenerated by the RegEx LDM 130 will reflect likelihood of correctnessand highlight London, UK, as the best, but if there is a ‘.uk’ at theend of the relevant hostname, then the location filter 122 can save thelocation synthesis process 124 from doing hundreds of thousands ofextraneous operations.

Location Synthesis Process (126)

The collection 120 of LDMs can conceptually be thought of as acollection of independent, artificially intelligent agents thatcontinuously look at data and use their respective artificialintelligences to make decisions. In the exemplary embodiment there arethus conceptually eight artificially intelligent agents mapping theInternet at relatively high speeds. An issue arises, however, in thatthere may be conflicts or disagreements in the results delivered by eachof these artificially intelligent agents.

The collection 120 of different LDMs may disagree on any number ofdifferent levels. For example, two LDMs may return the same country andregion, but different states and DMAs (Designated Marketing Areas).Alternatively, for example, one LDM may return a country only, whileanother LDM returns a city in a different country but on the samecontinent.

The unified mapping process 61, in one exemplary embodiment, includesthe ability to analyze where the incoming location determinants agree,and where they disagree. From this analysis, the unified mapping process61 operates to select the location determinant that has the highestlikelihood of being correct. In order to perform this selection, theunified mapping process 61 includes the capability to assess thelikelihood that it is correct.

To assist in the unified mapping process 61 with decision making, theLDMs provide associated confidence factors along with the locationdeterminants, as described above. The confidence factors comprisequantitative values indicating levels of confidence that the LDMs havethat the provided location determinants are in fact true. It should benoted that these confidence factors are not tied to any particular levelof geographic granularity (or geographic resolution). In one exemplaryembodiment of the present invention, the location synthesis process 124operates to produce a separate confidence factor for each level ofgeographic resolution or granularity (e.g., country, state, etc.).

FIG. 25 is a flowchart illustrating a method 380, according to anexemplary embodiment of the present invention, performed by the locationsynthesis process 124 to deliver a single location determinant which theunified mapping process 61 has identified as being the best estimate ofthe “true” geographic location associated with any particular networkaddress. An initial discussion provides a high-level overview of themethod 380, with further details being provided below in the context ofan illustrative example.

The method 380 commences at block 382, where the location synthesisprocess 124 compares every location determinant received from thelocation filter 122 against every other location determinant (whereappropriate). At block 384, the location synthesis process 124 builds aconfirmation confidence factor table. At block 386, the locationsynthesis process 124 collapses separate confidence factors into one ormore confirmation confidence factors, and at block 388 chooses a singlelocation determinant as the best estimate based on one or moreconfirmation confidence factors. The choice of the “best estimate”location determinant at block 388 is performed by identifying thelocation determinant that exhibits a highest degree of confidencefactor-weighted agreement with all the other location determinants. Afinal table of confidence factors generated for the “best estimate”location determinant is reflective of that agreement. The method 380then ends at block 390.

The location synthesis process 124 takes its input in the form ofmultiple sets of location determinants, as stated above. In oneexemplary embodiment, a distinction is made between this method and amethod of a flat set of all location determinants. The locationdeterminants are provided to the location synthesis process 124 asmultiple sets. The provision the location determinants in sets indicatesto the location synthesis process 124 which location determinants shouldbe compared against other. Specifically, efficiencies can be achieved byavoiding the comparison of location determinants within a common set,delivered from a common LDM.

To illustrate this issue, suppose that the RegEx LDM 130 extracts twostrings, one that yields twenty (20) location determinants, and anotherthat yields fifty (50). Also suppose that the LKH LDM 140 is able togenerate a location determinant. Accordingly, in this example, a totalof 71 location determinants require consideration by the locationsynthesis process 124. If the process 124 flatly compared all 71 againsteach other, this would result in (70+69+68+ . . . +3+2+1) 2485comparisons. If, however each location determinant of each set canignore all sibling location determinants of the same set, it will beappreciated that only (20*51+50*21+70) 2140 comparisons are required. Afurther advantage of considering LDMs in sets, in addition to thereduction in number of comparisons, is the set interpretation; locationdeterminants generated from the exact same source should not, in oneexemplary embodiment, be allowed to confirm one another.

Accordingly, at block 382 of the method 380 described above withreference to FIG. 25, the location synthesis process 124 iterativelycompares each location determinant of each set with each locationdeterminant of each other set. The comparison, in exemplary embodiment,because at a number of resolutions, for example:

-   -   1. Continent;    -   2. Country;    -   3. Region;    -   4. State;    -   5. DMA;    -   6. MSA;    -   7. PMSA; and    -   8. City.

These comparisons give rise to the confirmation confidence factor table,which is generated at block 384 of the method 380. The confirmationconfidence factor table is a matrix of location determinants bygeographic location resolution with their respective confirmationconfidence factor. The confirmation confidence factor calculation can beinterpreted as a calculation of the probability that any of the agreeinglocation determinants are correct, given that the associated confidencefactors are individual probabilities that each is independently correct.

An illustrative example of the calculation of the confirmationconfidence factor table, which uses a limited number of resolutionlevels and very few location determinants, is provided below. Table 4,below, illustrates an exemplary input of location determinants andassociated confidence factors provided to the location synthesis process124 from the location filter 122.

TABLE 4 Example input for the location synthesis process 124.Post-Filter Location Synthesis Process Input (Location Determinants andassociated Confidence Factors) Set 1 Set 2 Set 3 Set 4 New York, NY, USAElizabeth, NJ, London, Newark, NJ, USA [30] USA [25] UK [20] [50] NewYork (ST), USA [25]

In this example, there are four input sets, each with one or morelocation determinants and a confidence factor for each locationdeterminant. The initial (empty) confirmation confidence factor matrixtakes the form of the Table 5 illustrated below.

TABLE 5 Initial confirmation confidence factor matrix. Country StateCity New York, NY, USA New York State, USA Elizabeth, NJ, USA London, UKNewark, NJ, USA

Each element of the matrix is computed by comparing all relevant (nointra-set mingling) matches. For example, evaluating the countryconfidence factor for New York, N.Y., USA yields the following Table 6:

TABLE 6 Example Location Determinant Comparisons. Matches Country(always New York, NY, USA match self) Cannot Compare (same set) New YorkState, USA Matches Country Elizabeth, NJ, USA Does Not Match CountryLondon, UK Matches Country Newark, NJ, USA

In order to collapse of the separate confidence factors into a combinedconfidence factor, at block 386 of the method 380 illustrated in FIG.25, use is made of a confirmation confidence factor formula. An exampleof such a confirmation confidence factor formula is provided below:

If mcf_(i) is the i^(th) of n confidence factors from matching locationdeterminants, then the confirmation confidence factor (CCF) is computedby:${CCF} = {100 \times \left\lbrack {1 - {\prod\limits_{i = 1}^{n}\quad\left( {1 - \frac{{mcf}_{1}}{100}} \right)}} \right\rbrack}$

In the illustrative example, New York City matches with itself,Elizabeth, and Newark at the country level (e.g., a first level ofgeographic resolution). Accordingly, utilizing the above confirmationconfidence factor formula, the location synthesis process 124 combinesthese three associated confidence factors (30, 25, and 50) to deliverthe following confirmation confidence factor:CCF=100{1−[(1−0.30)(1−0.25)(1−0.50)]}CCF=73.75

Confirmation confidence factors are, in this way, generated at aplurality of geographic resolutions (e.g., continent, country, state,city) by detecting correspondences between the location determinants ateach of these geographic resolutions, and calculating the confirmationconfidence factors for each of these geographic resolutions for each ofthe location determinants. Accordingly, utilizing the about calculation,the confirmation confidence factor table illustrated in Table 6 ispopulated as illustrated below in Table 7:

TABLE 6 Completed confirmation confidence factor table. Country StateCity New York, NY, 73.75 30 30 USA New York State, 71.88 25 NA USAElizabeth, NJ, 80.31 62.5 25 USA London, UK 20 20 20 Newark, NJ, USA80.31 62.5 50

It will be noted that the “state” and “city” confirmation confidencefactors for the “New York, N.Y., USA” location determinant correspondedto the original, combined confirmation confidence factor (as generatedby a LDM) for this location determinant, in view of the absence of anycorrespondence, or agreement, at the “state” and “city” geographicresolution levels for this location determinant. On the other hand, astwo (2) agreement instances were detected for this location determinantat the “country” geographic resolution level, the confirmationconfidence factor at this geographic resolution is higher than theoriginal combined confirmation factor.

After the entire confirmation confidence factor table (or matrix) isgenerated at block 386, the location synthesis process 124 then has thetask of identifying the “best estimate” location determinant at block388. In the previous example, the correct answer is apparent from thecombined confidence factor table. There is no better choice than Newark,N.J.; it is tied for first place on country and state levels, but it isfirst at the city level. However, consider the more complex examples inwhich one location determinant has the highest state confidence factor,but another has the highest DMA (Designated Marketing Area) confidencefactor. To handle cases such as this, the location synthesis process 124generates a combined confirmation confidence factor that is a linearcombination of the constituent confirmation confidence factors.

For the purposes of generating the combined confirmation confidencefactor, different weights may, in an exemplary embodiment, be assignedto each of a plurality of levels of geographic resolution. Exemplaryweights that may be utilized in the linear combination of theconfirmation confidence factors are provided below:

1. City 30 2. State 20 3. Country 15 4. Region 10 5. MSA 0 6. PMSA 0 7.DMA 80 8. Continent 5

These exemplary weights are indicative of the importance andsignificance of agreement at a given level of geographic resolution. Forexample, the PMSA and MSA geographic resolutions each have a zero weightbecause of their close ties with the DMA and City geographicresolutions. Agreement at the continental geographic resolution level iscommon and easy to achieve, and this resolution level is weighted verylow in the combined confirmation confidence factor. Because the DMAgeographic resolution level is considered to be the most significantlevel in the exemplary embodiment, it is allocated the highest weight.

Any geographic resolution levels that are not available (e.g., foreigncountries do not have DMAs) are not utilized in the averaging process,and accordingly neither detriment nor assist the combined confirmationconfidence factor.

After the generation of the combined confirmation confidence factor, thelocation synthesis process 124 selects the largest valued combinedconfidence factor and uses that location determinant as the final result(i.e., the “best estimate” location determinant). The location synthesisprocess 124 returns the single “best estimate” location determinant,along with an associated LPT (Location Probability Table) thatconstitutes the relevant location determinant's row of the confirmationconfidence factor table.

In an exemplary embodiment of the present invention, an LPT table (notshown) is maintained within the data warehouse 30 and stores thelocation probability tables generated for a block of network addresses(or for an individual network address). An exemplary LPT table entry isprovided below as Table 7:

TABLE 7 LPT Column Description OCT1 1^(st) octet of the Network OCT22^(nd) octet of the Network OCT3 3^(rd) octet of the Network OCT4 4^(th)octet of the Network CONTINENT Continent code from the ContinentsReference CODE Table where the Network is located. CONTINENT ConfidenceFactor Associated with the CONFIDENCE Identified Continent. FACTORCOUNTRY Country code from the Countries Reference CODE Table where theNetwork is located. COUNTRY Confidence Factor Associated with theCONFIDENCE Identified Country. FACTOR REGION Region code from theRegions Reference Table CODE where the Network is located. This will beone of the Regions in the United States like Mid- West, West etc. REGIONConfidence Factor Associated with the CONFIDENCE Identified Region.FACTOR STATE CODE State code or equivalent like Province Code, from theStates Reference Table where the Network is located. STATE ConfidenceFactor Associated with the CONFIDENCE Identified State. FACTOR DMA CODEDesignated Market Area Code in United States where the network islocated. Applicable only for the networks in US DMA Confidence FactorAssociated with the CONFIDENCE Identified DMA FACTOR PMSA CODE PrimaryMetropolitan Statistical Area Code in United States where the network islocated. Applicable only for the networks in US. PMSA Confidence FactorAssociated with the CONFIDENCE Identified PMSA. FACTOR MSA CODEMetropolitan Statistical Area Code in United States where the network islocated. Applicable only for the networks in United States MSAConfidence Factor Associated with the CONFIDENCE Identified MSA. FACTORCITY CODE City code from the Cities Reference Table where The Network islocated CITY Confidence Factor Associated with the CONFIDENCE IdentifiedCity. FACTOR ZIP CODE ZIP CODE or equivalent of the location where thenetwork is located. ZIP Confidence Factor Associated with the CONFIDENCEIdentified ZIP CODE FACTOR AREA CODE Telephone Area Code of the locationwhere the network is located. Applicable to United States networks. AREACODE Confidence Factor Associated with the CONFIDENCE Identified AREACODE FACTOR LATUTUDE Latitude of the location where the network islocated. LONGITUDE Longitude of the location where the network islocated. TIMEZONE Time Zone of location where the network is located.

Confidence Accuracy Translator (126)

In one exemplary embodiment, in order to assist in the interpretation ofthe end data, the unified mapping process 61 outputs the “best estimate”location determinant together with a full Location Probability Table(LPT) (i.e., the end result 128 illustrated in FIG. 11). The values ofthe location probability table are the probabilities that the givenlocation is correct at a number of geographic location resolution levels(or granularities). The location synthesis process 124 does return anapplication probability table, and while the values in that areself-consistent and relatively meaningful, they are not locationprobabilities in the formal sense.

In the exemplary embodiment, a translation is provided so that when acustomer gets a result that is reported with a “90” confidence factor,the customer can know that if 100 records all with 90 confidence factorwere pulled at random, roughly 90 of them would be correct. Thistranslation function is performed by the confidence accuracy translator126.

Accuracy cannot be inferred by a single observation. A singleobservation is either right or wrong. It is only by looking at aggregatecorrectness that assertions can be made about accuracy.

FIG. 26 is a graph 400 illustrating correctness of locationdeterminants, as a function of post-location synthesis processconfidence factor. It will be noted from the graph 400 that, in general,incorrect responses are generally given low confidence factors, and thehigher confidence factors are generally associated with morecorrectness. To formalize this relationship, a moving average can beused to infer the rough relationship between confidence factors andaccuracy.

FIG. 27 is a graph 402 illustrating correctness of location determinantsas a function of post-LSP confidence factor, and the smoothedprobability of correctness given a confidence factor range. In FIG. 27,a curve 404 is a 41-point moving average, representing the probabilitythat the given responses in that confidence factor neighborhood areright. Again, it has the desired shape. Low confidence factors areassociated with low accuracy, and conversely, high confidence factorsare associated with high accuracy. Through this, it is clear thatcarrying the confidence factors throughout the unified mapping process61 is beneficial, because, in this way, not only can the unified mappingprocess 61 generally be skillful, but it can know when it is lessskillful. What remains, however, it the final translation ofpost-location synthesis process confidence factors intoprobabilistically meaningful confidence factors.

This translation is represented by the curve 404 of FIG. 27. To avoidover-fitting to the noise of the function, the confidence accuracytranslator 126 uses a piecewise linear approximation of the function bybinning the data into equally sized, disjoint confidence factor bins.

FIG. 28 is a graph 406 illustrating correctness of location determinantsas a function of post-LSP confidence factor, and the smoothedprobability of correctness given a confidence factor range with picewiselinear approximation. As shown in FIG. 28, a curve 408 is theapproximation of the confidence factor-Accuracy relationship generatedwith each abscissa being the average confidence factor of the bin andeach ordinate being the number of accuracy within the bin. Accordingly,the curve 408 can be and is used as an interpolation scheme for unifiedmapping process 61 to make the needed translation.

While interpolation is a fairly low-risk method for inferringinformation, extrapolation can provide incorrect data. Note from FIG. 28that there is insufficient data with confidence factor less than 20 orgreater than 65 to establish a significant relationship. Yet, therequired robust translation must account for any confidence factor inthe valid range of 0 to 100. In this way, the confidence accuracytranslator 126 is forced to extrapolate, but does so in a restraintmanner. Erring on the side of less expected accuracy, the confidenceaccuracy translator 126 introduces two new points to the interpolationscheme: [0,0], and [100,max(CF_(avg))]. This implies that if thelocation synthesis process 124 returns with a zero confidence factor, itis incorrect and that if it returns with any confidence factor greaterthan the maximum of the binned interpolation nodes, then it hasprecisely the same accuracy as the best bin.

These artificial extrapolations (shown at 410 in FIG. 28) will make theaccuracy over the unified mapping process 61 appear lower than it reallyis. Combining the curves 408 and 410, the entire set of confidencefactors can now be translated. This translation is illustrated in FIG.29. More specifically, FIG. 29 shows a graph 411 plotting correctness oflocation determinants as a function of post-CAT confidence factor, andthe smoothed probability of correctness. Final results of the post-CATconfidence factors are compared against the actual accuracy in FIG. 29.As can be noted, there is a strong correlation, thus giving the finalconfidence factor the probabilistic meaning that is useful to end usersto make meaningful decisions. While there is strong correlation, itshould be noted that this is a general relationship and that, whilepulling a random subset and verifying should yield comparable results,data may be noisy, and some populations may show disparities betweenconfidence and real accuracy.

A number of further algorithms are now described. These furtheralgorithms may be deployed in alternative embodiments of the presentinvention, and in conjunction with any of the algorithms (e.g., LDMs)discussed above.

Latitude and Longitude Matching

In one embodiment of the present invention, a latitude and longitudematching process may be utilized used to assist in the determination thegeographic location of a given record. Only a network address (e.g., andIP address) is required for the longitude and latitude matching processto be successful. However, additional information, such as the owner'slocation, or proximal routers, may be utilized to achieve a higherprobability of success.

The geographic locations identified by the longitude and latitudematching is utilized to compute distances, using this information todetermine accuracy of a given record. The information is compared withprevious “hops” of the traceroute to the host. If the route forms apredictable pattern, a confidence factor maybe be increased.

Launching traces from network and geographically disperse locations,algorithms may compute the similarity of each trace, arriving at a finalconfidence factor ranking. The higher the ranking, the more likely thelocation attempt was successful.

EXAMPLE 1

The last four hops in a traceroute form a distal-proximal relationship,meaning that the next hop is geographically closer to its nextsuccessive hop:

-   Hop 5 is closer to hop 6-   Hop 6 is closer to hop 7-   Hop 7 is closer to hop 8

Thus, the traced route geographically progresses toward the final hop 8,leading to a decision that the destination is located within a certainrange of accuracy.

EXAMPLE 2

The point of origin is Denver, Colo., and the destination is Salt LakeCity, Utah. The last four hops indicate a connection that is back-hauledthrough Denver, Colo., essentially geographically backtracking the routetaken:

-   1 Denver Router-   2 Grand Junction Router-   3 Provo Utah Router-   4 Salt Lake City Router-   5 Salt Lake City Router-   6 Denver Router-   7 Provo Utah Router-   8 Salt Lake City Router-   9 Salt Lake City Destination

This example indicates a geographic progression away from Denver towardUtah, directly back to Denver, and finally directly back to Utah with adestination that does not leave Utah. Thus, a human may assume that eventhough the route taken was very indirect, it did terminate in Utah.Using Latitude/Longitude coordinates, the data collection agents 18 willsee the same scenario and arrive at an intelligent conclusion.

Triangulation

Using a translation process, in one exemplary embodiment of the presentinvention, an approximate radius containing the target network addressbe generated. Launching a latitude/longitude route discovery fromgeographically disperse locations, the final destination will likelyproceed through the same set of routers. Thus, if the final 3 hopsleading up to the point of entry into the destination network areproximal, or at the very least, form a line toward the destination'spoint of entry, one may assume that the destination resides within thecommon latitude/longitude coordinates. Using the attitude/latitudecoordinates of other known landmarks allows a radius to be computed.Within this radius, metro areas and large cities will be known.

Example

A traceroute is launched from the East Coast, the West Coast, and theNorth West. Route progression from the East Coast indicates a westwardpath, terminating in Texas. Route progression from the West Coastindicates an eastward path, terminating in Texas. Route progression fromthe North West indicates an eastward path, terminating in Texas.

Being that all routes terminated in Texas, and the associated record forthe target indicates a Texas-based owner, specifically, Dallas, one mayassume that in fact, the target resides in the DFW metro area.

Triangulation is the technique of using traceroutes originating fromgeographically widely separated locations and using the results toextrapolate a possible location for the target network address.

Once all the traceroutes have been completed, a general direction (e.g.Northward, Eastward) may be extrapolate from the traceroutes usingknowledge of the locations of the routers in the traceroute. This canthen be used to place bounds on the possible location by creating anintersection of all traceroutes. For example, a traceroute going Eastfrom San Francisco, West from New Jersey is probably somewhere in theCentral time zones. Directions for the traceroutes can be inferred bysubtracting the geographical locations of the originating networkaddress from those of the latest router in the trace that has a knownlocation. Additionally, information about the number of hops in thetraceroutes can be used to obtain estimates of distance.

Because a number of traceroutes should be obtained for each targetnetwork address, an infrastructure is in place to distribute theserequests. One exemplary manner of implementing the system is to have asingle script on a single machine make “rsh” calls to remote machines toobtain the traceroutes. This avoids they need for buffering andsynchronization (these are pushed off to the operating system calls thatimplement the blocking for the rsh command). The machines used mayactually be the same machines as used for the dialup method. These arealready connected to ISPs at widely separated locations.

In addition to a confidence factor, a translation process may alsogenerate a resolution indication. This will depend on:

If all the traces seem to be going in the same direction. If so theresolution is low (do the trigonometry).

The number of traces available. The more traces, the higher theresolution.

The variance in the distances obtained. Each trace will result in acircle around the predicted point according to the expected variance inthe distance. The intersection of these circles dictates the probablelocation. The area of the intersection dictates the resolution (thelarger the area the lower the resolution). The distance scale and thevariances can only be calibrated using experimental results from knownlocations.

Computer System

FIG. 30 shows a diagrammatic representation of machine in the exemplaryform of a computer system 500 within which a set of instructions, forcausing the machine to perform any one of the methodologies discussedabove, may be executed. In alternative embodiments, the machine maycomprise a network router, a network switch, a network bridge, PersonalDigital Assistant (PDA), a cellular telephone, a web appliance or anymachine capable of executing a sequence of instructions that specifyactions to be taken by that machine.

The computer system 500 includes a processor 502, a main memory 504 anda static memory 506, which communicate with each other via a bus 508.The computer system 500 may further include a video display unit 510(e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). Thecomputer system 500 also includes an alpha-numeric input device 512(e.g. a keyboard), a cursor control device 514 (e.g. a mouse), a diskdrive unit 516, a signal generation device 518 (e.g. a speaker) and anetwork interface device 520.

The disk drive unit 516 includes a machine-readable medium 522 on whichis stored a set of instructions (i.e., software) 524 embodying any one,or all, of the methodologies described above. The software 524 is alsoshown to reside, completely or at least partially, within the mainmemory 504 and/or within the processor 502. The software 524 may furtherbe transmitted or received via the network interface device 520. For thepurposes of this specification, the term “machine-readable medium” shallbe taken to include any medium which is capable of storing or encoding asequence of instructions for execution by the machine and that cause themachine to perform any one of the methodologies of the presentinvention. The term “machine-readable medium” shall accordingly be takento included, but not be limited to, solid-state memories, optical andmagnetic disks, and carrier wave signals.

Thus, a method and system to modify geolocation activities based onlogged query information have been described. Although the presentinvention has been described with reference to specific exemplaryembodiments, it will be evident that various modifications and changesmay be made to these embodiments without departing from the broaderspirit and scope of the invention. Accordingly, the specification anddrawings are to be regarded in an illustrative rather than a restrictivesense.

1. A method to perform geolocation activities relating to a networkaddress, the method including: maintaining a database of networkaddresses and associated geographic locations; receiving a query,including a network address, against the database for a geographiclocation associated with the network address; logging informationconcerning the query received against the database; and modifyinggeolocation activities relating to at least the network address based onthe logged information, wherein the geolocation activities include:collecting network information pertaining to at least the networkaddress; and estimating the geographic location associated with thenetwork address, based on the collected network information, and whereinthe modifying of the geolocation activities includes: prioritizing thegeolocation activities relating to at least the network address.
 2. Themethod of claim 1, wherein the logging of information concerning thequery includes logging the network address if a record identifying thegeographic location associated with the network address is not locatedwithin the database.
 3. The method of claim 1, wherein the collecting ofthe network information pertaining to at least the network addressincludes collecting the geolocation information utilizing a plurality ofdata collection agents.
 4. The method of claim 3, wherein the pluralityof data collection agents are geographically dispersed.
 5. The method ofclaim 1, wherein the collecting of the network information is performedutilizing a plurality of data collection processes.
 6. The method ofclaim 1, wherein the collecting of the network information is performedfrom a plurality of data sources.
 7. The method of claim 1, includingdetermining that the database does not indicate a geographic location asbeing associated with the network address, and wherein the modifyingincludes prioritizing the geolocation activities relating to the networkaddress based on the determination that the database does not indicatethe geographic location as being associated with the network address. 8.The method of claim 7, including maintaining a log of network addressesfor which the database does not indicate respective geographic locationsas being associated with the network addresses, and prioritizing thegeolocation activities based on the log.
 9. The method of claim 8,wherein the log is a customer access log that indicates location misses,the method including obtaining information concerning location missesfrom the customer access log.
 10. The method of claim 8, wherein theprioritizing of the geolocation activities is with respect to aplurality of network addresses included in the log.
 11. The method ofclaim 7, including receiving the query from a requester, andcommunicating a message to the requester indicating an absence of anassociation between the network address and the geographic location inthe database.
 12. The method of claim 1, wherein the query is receivedfrom a customer website, responsive to a user accessing the customerwebsite, the network address being the network address associated with amachine of the user.
 13. The method of claim 1, wherein the query isreceived via an Application Program Interface (API).
 14. The method ofclaim 1, wherein the query is received via a customer extranet.
 15. Themethod of claim 6, wherein the estimating includes determining whetherthe network address falls within a consolidated domain of networkaddresses maintained within the database.
 16. The method of claim 15,wherein the consolidated domain of network addresses maintained withinthe database includes any one of a group of domains including aneducational, business, service provider and government domain.
 17. Themethod of claim 6, wherein the estimating includes identifying a networkaddress block around the network address.
 18. The method of claim 6,wherein the estimating includes running an exact geolocation process todetermine geolocation information for the network address.
 19. Themethod of claim 17, wherein the estimating includes running an exactgeolocation process to determine geolocation information for theidentified network address block around the network address.
 20. Themethod of claim 18, wherein the exact geolocation process includes anyone of a group of geolocation processes including a traceroute, alatency calculation, a hostname matching operation and a DNS process.21. The method of claim 6, wherein the estimating includes running aninexact geolocation process to determine geolocation information for thenetwork address.
 22. The method of claim 6, wherein the estimatingincludes forwarding the network address for manual resolution.
 23. Themethod of claim 6, wherein the estimating is a tiered process, includinga plurality of sequential automated mapping operations to associate thegeographic location with the network address.
 24. A system to performgeolocation activities relating to a network address, the systemincluding: a database of network addresses and associated geographiclocations; and a server to: receive a query, including a networkaddress, against the database for a geographic location associated withthe network address; log information concerning the query receivedagainst the database; and modify geolocation activities relating to atleast the network address based on the logged information, wherein theserver is to perform geolocation activities including: collectingnetwork information pertaining to at least the network address; andestimating the geographic location associated with the network address,based on the collected network information, and wherein the server is tomodify the geolocation activities by: prioritizing the geolocationactivities relating to at least the network address.
 25. The system ofclaim 24, wherein the server is to log the network address if a recordidentifying the geographic location associated with the network addressis not located within the database.
 26. The system of claim 24,including a plurality of data collection agents to collect thegeolocation information.
 27. The system of claim 26, wherein theplurality of data collection agents are geographically dispersed. 28.The system of claim 24, wherein the collecting of the networkinformation is perfonned utilizing a plurality of data collectionprocesses.
 29. The system of claim 24, wherein the collecting of thenetwork information is performed from a plurality of data sources. 30.The system of claim 24, wherein the server is to determine that thedatabase does not indicate the geographic location as being associatedwith the network address, and to prioritize the geolocation activitiesrelating to the network address based on the determination that thedatabase does not indicate the geographic location as being associatedwith the network address.
 31. The system of claim 30, including a log ofnetwork addresses for which the database does not indicate respectivegeographic locations as being associated with the network addresses, theserver to prioritize the geolocation activities based on the log. 32.The system of claim 31, wherein the log is a customer access log thatindicates location misses, the server to obtain information concerninglocation misses from the customer access log.
 33. The system of claim31, wherein the server is to prioritize the geolocation activities withrespect to a plurality of network addresses included in the log.
 34. Thesystem of claim 24, wherein the server is to receive the query from arequester, and to communicate a message to the requester indicating anabsence of an association between the network address and the geographiclocation in the database.
 35. The system of claim 24, wherein the queryis received at the server from a customer website, responsive to a useraccessing the customer website, the network address being the networkaddress associated with a machine of the user.
 36. The system of claim24, wherein the query is received via an Application Program Interface(API).
 37. The system of claim 24, wherein the query is received via acustomer extranet.
 38. The system of claim 24, wherein the server is todetermine whether the network address is likely to fall within aconsolidated domain of network addresses maintained within the database.39. The system of claim 24, wherein the consolidated domain of networkaddresses maintained within the database includes any one of a group ofdomains including an educational, business, service provider andgovernment domain.
 40. The system of claim 24, wherein the server is toidentify a network address block around the network address.
 41. Thesystem of claim 24, wherein the estimating includes running an exactgeolocation process to determine geolocation information for the networkaddress.
 42. The system of claim 24, wherein the estimating includesrunning an exact geolocation process to determine geolocationinformation for the identified network address block around the networkaddress.
 43. The system of claim 42, wherein the exact geolocationprocess includes any one of a group of geolocation processes including atraceroute, a latency calculation, a hostname matching operation and aDNS process.
 44. The system of claim 24, wherein the estimating includesrunning an inexact geolocation process to determine geolocationinformation for the network address.
 45. The system of claim 24, whereinthe estimating includes forwarding the network address for manualresolution.
 46. The system of claim 24, wherein the estimating is atiered process, including a plurality of sequential automated mappingoperations to associate the geographic location with the networkaddress.
 47. A machine-readable medium storing a set of instructionsthat, when executed by a machine, cause the machine to perform a methodto perform geolocation activities relating to a network address, themethod including: maintaining a database of network addresses andassociated geographic locations; receiving a query, including a networkaddress, against the database for a geographic location associated withthe network address; logging information concerning the query receivedagainst the database; and modifying geolocation activities relating toat least the network address based on the logged information, whereinthe geolocation activities include: collecting network informationpertaining to at least the network address; and estimating thegeographic location associated with the network address, based on thecollected network information, and wherein the modifying of thegeolocation activities includes: prioritizing the geolocation activitiesrelatina to at least the network address.
 48. A system to performgeolocation activities relating to a network address, the systemincluding: first means for storing network addresses and associatedgeographic locations; and second means for: receiving a query, includinga network address, against the first means, the query being for ageographic location associated with the network address; determiningthat the first means does not indicate a geographic location as beingassociated with the network address; logging information concerning thequery received against the first means; and modifying geolocationactivities relating to at least the network address based on the loggedinformation, wherein the second means is for performing aeolocationactivities including: collecting network information pertaining to atleast the network address; and estimating the geographic locationassociated with the network address, based on the collected networkinformation, and wherein the second means is for modifying thegeolocation activities by: prioritizing the geolocation activitiesrelating to at least the network address.